System and method for computer aided diagnosis

ABSTRACT

The present disclosure relates to a method for training a classifier. The method includes: acquiring an original image; determining a candidate target by segmenting the original image based on at least two segmentation models; determining a universal set of features by extracting features from the candidate target; determining a reference subset of features by selecting features from the universal set of features; and determining a classifier by performing classifier training based on the reference subset of features.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Application No.PCT/CN2016/107278, filed on Nov. 25, 2016, which claims priority ofChinese Patent Application No. 201510862056.2 filed on Nov. 30, 2015,and Chinese Patent Application No. 201610283527.9 filed on Apr. 29,2016, the entire contents of which are hereby incorporated by reference.

TECHNICAL FIELD

The present disclosure generally relates to systems and methods forcomputer aided diagnosis (CAD), and more particularly to systems andmethods for segmenting an image and training a classifier in computeraided diagnosis.

BACKGROUND

The computer aided diagnosis (CAD) system may be used to detect lesions,display diagnosis results relating to the lesions to doctors, andsupport doctors to locate, diagnose and quantitatively analyze thelesions (e.g., a pulmonary nodule, a breast calcification, a breastmass, polyp, etc.) in medical images to reduce misdiagnoses and misseddiagnoses and increase correct diagnosis rate. Accordingly, it would bedesirable to improve the efficiency and accuracy of the detection and/ordiagnosis of a CAD system.

SUMMARY

According to a first aspect of the present disclosure, a method fortraining a classifier may comprise: acquiring an original image;determining a candidate target by segmenting the original image based onat least two segmentation models; determining a universal set offeatures by extracting features from the candidate target; determining areference subset of features by selecting features from the universalset of features; and determining a classifier by performing classifiertraining based on the reference subset of features.

In some embodiments, the original image may include an X-ray image, acomputer tomography (CT) image, a Positron Emission Tomography (PET)image, a Magnetic Resonance Imaging (MRI) image, an ultrasonic image, anelectrocardiogram, or an electroencephalogram.

In some embodiments, the at least two segmentation models may include amorphological model and a statistical model.

In some embodiments, the morphological model may include a fixedthreshold region growing model based on a Hessian enhancement.

In some embodiments, the statistical model may include a clusteringmodel.

In some embodiments, the clustering model may include a variationalexpectation maximization model.

In some embodiments, the method may further comprise: determiningpreprocessed data by preprocessing the original image, the preprocessingincluding an interpolation treatment, a morphological treatment, aHessian dot-enhancement treatment, or a Hessian line-enhancementtreatment.

In some embodiments, the universal set of features may include featuresextracted from the candidate target, the candidate target may bedetermined by segmenting the preprocessing data based on the at leasttwo segmentation models.

In some embodiments, the method of determining the reference subset offeatures by selecting features from the universal set of features mayinclude a simulated annealing algorithm.

According to a second aspect of the present disclosure, a method ofcomputer aided diagnosis may comprise: acquiring an original image;determining a candidate target by segmenting the original image based onat least two segmentation models; acquiring a classifier including areference subset of features; determining feature data by extractingfeatures from the candidate target based on the reference subset offeatures; and determining a classification result by classify thecandidate target based on the classifier and the feature data.

In some embodiments, the original image may include an X-ray image, acomputer tomography (CT) image, a Positron Emission Tomography (PET)image, a Magnetic Resonance Imaging (MRI) image, an ultrasonic image, anelectrocardiogram, or an electroencephalogram.

In some embodiments, the determining of the candidate target bysegmenting the original image based on the at least two segmentationmodels may comprise: determining one or more first positioning regionsby performing preliminary positioning on the original image; determininga second positioning region by performing threshold segmentation on theone or more first positioning regions based on Hessian dot-enhancement,the second positioning region including the candidate target and abackground region, the candidate target including a substance region anda surrounding region that surrounds the substance region; determiningthe substance region of the candidate target based on the secondpositioning region and a statistical model; determining the surroundingregion of the candidate target based on the statistical model and amorphological model; and determining the candidate target by combiningthe substance region and the surrounding region.

In some embodiments, the determining of the one or more firstpositioning regions by performing the preliminary positioning on theoriginal image may comprise: receiving information regarding the one ormore first positioning region from a use; determining, based on thereceived information, the one or more first positioning regions;determining the one or more first positioning regions using a regiongrowing method by determining an axis that goes through the candidatetarget and selecting a seed point from the axis; or determining the oneor more first positioning regions based on a detection processing.

In some embodiments, the statistical model may include a variationalexpectation maximization model.

In some embodiments, the determining of the substance region of thecandidate target based on the second positioning region and thestatistical model may comprise: processing, based on the variationalexpectation maximization model, the second positioning region todetermine a probability graph; determining a pixel or voxel in thesecond positioning region corresponding to a pixel or voxel in theprobability graph; determining that a probability value of the pixel orvoxel is greater than the first threshold and the gray value of thepixel or voxel is greater than the second threshold; determining, inresponse to the determination the probability value of the pixel orvoxel is greater than the first threshold, and the gray value of thepixel or voxel is greater than the second threshold, the pixel or voxelbelongs to the substance region; determining that a probability value ofthe pixel or voxel is equal to or less than the first threshold, and agray value of the pixel or voxel is equal to or less than the secondthreshold; and determining, in response to the determination theprobability value of the pixel or voxel is equal to or less than thefirst threshold, and the pixel or voxel belongs to the backgroundregion.

In some embodiments, the morphological model may include a Hessianline-enhancement model.

In some embodiments, the determining of the surrounding region maycomprise: processing the second positioning region, based on the Hessianline-enhancement model to determine a Hessian line-enhancement image;determining a pixel or voxel in the Hessian line-enhancement imagecorresponding to a pixel or voxel in the probability graph; determiningthat a probability value of the pixel or voxel is greater than the firstthreshold and a gray value of the pixel or voxel is less than the thirdthreshold; determining, in response to the determination that theprobability value of the pixel or voxel is greater than the firstthreshold and the gray value of the pixel or voxel is less than thethird threshold, the pixel of voxel belongs to the surrounding region;determining that a probability value of the pixel or voxel is equal toor less than the first threshold and a gray value of the pixel or voxelis equal to or greater than the third threshold; and determining, inresponse to the determination that the probability value of the pixel orvoxel is equal to or less than the first threshold and the gray value ofthe pixel or voxel is equal to or greater than the third threshold, thepixel of voxel belongs to the background region.

In some embodiments, spatial positions of pixels or voxels in theprobability graph, the second positioning region, and the Hessianline-enhancement image may be corresponding.

According to a third aspect of the present disclosure, a system oftraining a classifier may comprise: an original data acquisition unitconfigured to acquire an original image; a candidate targetdetermination unit configured to determine a candidate target bysegmenting the original image based on at least two segmentation models;a first feature extraction unit configured to determining a universalset of features by extracting features from the candidate target; afeature selection unit configured to determining a reference subset offeatures by selecting features from the universal set of features; and atraining unit configured to determining a classifier by performingclassifier training based on the reference subset of features.

According to a fourth aspect of the present disclosure, a system ofcomputer aided diagnosis may comprise: an original data acquisition unitconfigured to acquire an original image; a candidate targetdetermination unit configured to determine a candidate target bysegmenting the original image based on at least two segmentation models;a classifier acquisition unit configured to acquire a classifierincluding a reference subset of features; a second feature extractionunit configured to determine feature data by extracting features fromthe candidate target based on the reference subset of features; and aclassification unit configured to determine a classification result byclassify the candidate target based on the classifier and the featuredata.

According to a fifth aspect of the present disclosure, a system ofcomputer aided diagnosis may comprise: a message server clusterconfigured to distribute pending data; and a plurality of working nodesof a computer aided diagnosis server configured to acquire the pendingdata from the message server cluster and generate a processing result byprocessing the pending data in a streaming parallel method in real-time.

In some embodiments, the message server cluster may include a Kafkacluster.

In some embodiments, the pending data may be processed by a Stormcluster.

In some embodiments, the system may further comprise: a clustermanagement node configured to manage the Kafka cluster and the pluralityof working nodes, wherein the cluster management node includes aZookeeper cluster.

In some embodiments, the pending data may include an original medicalimage or intermedia data, and the intermedia data may include a regionof interest (ROI), preprocessed data, a candidate target, or featuredata.

According to a sixth aspect of the present disclosure, a method ofcomputer aided diagnosis may comprise: managing a message server clusterand a plurality of working nodes of a computer aided diagnosis server toperform operations including: distributing pending data through themessage server cluster in a streaming mode, wherein the pending dataincludes a region of interest (ROI), preprocessed data, a candidatetarget, or feature data; acquiring the pending data from the messageserver cluster through the plurality of working nodes of the computeraided diagnosis server; and processing the pending data in a streamingparallel mode in real time to generate a processing result.

According to a seventh aspect of the present disclosure, a medicalsystem may comprise: an input device configured to acquire medical data;a computer aided diagnosis device configured to acquire the medical datafrom the input device, and an output device configured to output theprocessing result. The computer aided diagnosis device may comprise: amessage server cluster configured to acquire the medical data from theinput device; and a plurality of working nodes of a computer aideddiagnosis server configured to acquire the pending data from the messageserver cluster and generate a processing result by processing thepending data in a streaming parallel method in real-time.

In some embodiments, the input device may include an imaging deviceconfigured to generate medical images, or a storage device configured tostore data from the imaging device or data from the working nodes, thestorage device includes at least one of a database, a PACS, or a filedevice.

In some embodiments, the system may further comprise: a clustermanagement node configured to manage the Kafka cluster and the pluralityof working nodes, wherein the cluster management node includes aZookeeper cluster.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is further described in terms of exemplaryembodiments. These exemplary embodiments are described in detail withreference to the drawings. These embodiments are non-limiting examples,in which like reference numerals represent similar structures throughoutthe several views of the drawings, and wherein:

FIG. 1 is a schematic diagram of an exemplary computer aided diagnosis(CAD) system according to some embodiments of the present disclosure;

FIG. 2-A is a block diagram illustrating an example of a computer aideddiagnosis (CAD) system according to some embodiments of the presentdisclosure;

FIG. 2-B is a flowchart illustrating an exemplary process for a computeraided diagnosis (CAD) according to some embodiments of the presentdisclosure;

FIG. 3 is a block diagram illustrating an example of a processing modulein the CAD system according to some embodiments of the presentdisclosure;

FIG. 4-A is a flowchart illustrating an exemplary process fordetermining a candidate target by a processing module according to someembodiments of the present disclosure;

FIG. 4-B is a flowchart illustrating an exemplary process for training aclassifier by a processing module according to some embodiments of thepresent disclosure;

FIG. 4-C is a flowchart illustrating an exemplary process for performingclassification based on a classifier by a processing module according tosome embodiments of the present disclosure;

FIG. 5 is a block diagram illustrating an example of a candidate targetdetermination unit according to some embodiments of the presentdisclosure;

FIG. 6 is a flowchart illustrating an exemplary process for determininga candidate target by a candidate target determination unit according tosome embodiments of the present disclosure;

FIG. 7 is a flowchart illustrating an exemplary process for determininga candidate target by a candidate target determination unit according tosome embodiments of the present disclosure;

FIG. 8 is a block diagram illustrating an example of a candidate targetdetermination unit according to some embodiments of the presentdisclosure;

FIG. 9 is a flowchart illustrating an exemplary process for determininga reference subset of features according to some embodiments of thepresent disclosure;

FIG. 10 is a schematic diagram for illustrating exemplary featuresources of feature extraction according to some embodiments of thepresent disclosure;

FIG. 11 is a flowchart illustrating an exemplary process for determininga reference subset of features based on a simulated annealing algorithmaccording to some embodiments of the present disclosure;

FIG. 12 is a flowchart illustrating an exemplary process for segmentinga region in real time according to some embodiments of the presentdisclosure;

FIG. 13 is a schematic diagram illustrating an exemplary segmentedresult based on a real-time segmentation technique according to someembodiments of the present disclosure;

FIG. 14-A and FIG. 14-B are schematic diagrams illustrating exemplarysegmented results based on a real-time segmentation technique accordingto some embodiments of the present disclosure;

FIG. 15 is a block diagram illustrating an example of a real-timestreaming parallel computing mode according to some embodiments of thepresent disclosure;

FIG. 16-A is an exemplary original chest computed tomography (CT) imageobtained through scanning a human body by a computed tomography (CT)device according to some present disclosure;

FIG. 16-B is an exemplary mask image relating to a pulmonary parenchymaaccording to some present disclosure;

FIG. 16-C is an exemplary segmented result relating to a pulmonarynodule according to some present disclosure;

FIG. 17 is a schematic diagram illustrating an example of a diagnosisresult generated by a computer aided diagnosis (CAD) system according tosome present disclosure; and

FIGS. 18-A and 18-B are exemplary images of a breast lump regionsegmented by a region real-time segmentation algorithm according to somepresent disclosure.

DETAILED DESCRIPTION

In order to illustrate the technical solutions related to theembodiments of the present disclosure, brief introduction of thedrawings referred to the description of the embodiments is providedbelow. Obviously, drawings described below are only some examples orembodiments of the present disclosure. Those having ordinary skills inthe art, without further creative efforts, may apply the presentdisclosure to other similar scenarios according to these drawings. Itshould be noted that the embodiments in this disclosure are provided forthe purpose of illustration, and not intended to limit the scope of thepresent disclosure. Unless stated otherwise or obvious from the context,the same reference numeral in the drawings refers to the same structureand operation.

The terminology used herein is for the purposes of describing particularexamples and embodiments only, and is not intended to be limiting. Asused herein, the singular forms “a,” “an,” and “the” may be intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “include,”and/or “comprise,” when used in this disclosure, specify the presence ofintegers, devices, behaviors, stated features, steps, elements,operations, and/or components, but do not exclude the presence oraddition of one or more other integers, devices, behaviors, features,steps, elements, operations, components, and/or groups thereof.

Some modules of the system may be referred to in various ways accordingto some embodiments of the present disclosure, however, any number ofdifferent modules may be used and operated in a client terminal and/or aserver. These modules are intended to be illustrative, not intended tolimit the scope of the present disclosure. Different modules may be usedin different aspects of the system and method.

According to some embodiments of the present disclosure, flow charts areused to illustrate the operations performed by the system. It is to beexpressly understood, the operations above or below may or may not beimplemented in order. Conversely, the operations may be performed ininverted order, or simultaneously. Besides, one or more other operationsmay be added to the flowcharts, or one or more operations may be omittedfrom the flowchart.

Embodiments in the disclosure may be applied to a computer aideddiagnosis (CAD) system. The CAD system may be used to detect lesions,display diagnosis results relating to the lesions to doctors, andsupport doctors to locate, diagnose and quantitatively analyze thelesions (e.g., a pulmonary nodule, a breast calcification, polyp, etc.)in medical images to reduce misdiagnoses and missed diagnoses andincrease diagnose accuracy rate. Systems and methods for CAD describedin the disclosure may be used by various imaging examination techniques,such as a computer tomography (CT), a magnetic resonance Imaging (MRI),a positron emission tomography (PET), an ultrasonic diagnosis, a plainfilm diagnosis, an X-ray imaging, an electrocardiograph (ECG) diagnosis,an electroencephalo-graph (EEG) diagnosis, etc. Systems and methods forCAD described in the disclosure may be used to detect and diagnose abreast, a chest, a lung nodule, a liver disease, a brain tumor, a colon,etc.

FIG. 1 is a schematic diagram of an exemplary computer aided diagnosis(CAD) system 100 according to some embodiments of the presentdisclosure. As shown, the CAD system 100 may include one or more imagingdevice 110, storage device 120, CAD server 130, and external device 140.

In some embodiments, the imaging device 110 may generate a medial image.In some embodiments, the medical image may be transmitted to the CADserver 130 for further processing, or stored in the storage device 120.The imaging device 110 may include a computer tomography (CT) device, amagnetic resonance Imaging (MRI) device, a positron emission tomography(PET) device, an ultrasound imaging device, an X-ray imaging device, anECG device, an EEG device, etc. The ultrasound imaging device mayinclude a B-scan ultrasonography device, a color Doppler ultrasounddevice, a cardiac color ultrasound device, a three-dimensional colorultrasound device, etc.

In some embodiments, the storage device 120 may include any device witha function of storing. In some embodiments, the storage device 120 maystore data acquired from the imaging device 110 (e.g., a medical imagegenerated by the imaging device 110), or various data generated by theCAD server 130. In some embodiments, the storage device 120 may belocal, or remote. The storage device 120 may include a database 120-1,picture archiving and communication systems (PACS) 120-2, a file device120-3, or the like, or a combination thereof. The database 120-1 mayinclude a hierarchical database, a network database, a relationaldatabase, or the like, or a combination thereof. The storage device 120may digitize information and store the digitized information in anelectric storage device, a magnetic storage device, an optical storagedevice, etc. The storage device 120 may store various information, suchas procedures, data, etc. The storage device 120 may be a device thatstores information using electric energy, such as a memorizer, a randomaccess memory (RAM), a read only memory (ROM), or the like, or acombination thereof. The RAM may include a dekatron, a selectron, adelay line memory, a Williams tube, a dynamic random access memory(DRAM), a static random access memory (SRAM), a thyristor random accessmemory (T-RAM), a zero capacitor random access memory (Z-RAM), or thelike, or a combination thereof. The ROM may include a read-only memorybubble memory, a magnetic button line memory, a memory thin film, amagnetic plate line memory, a core memory, a magnetic drum memory, aCD-ROM drive, a hard disk, a magnetic tape, an early nonvolatile memory(the NVRAM), a phase change memory, a magneto resistive random accessmemory modules, a ferroelectric random access memory, a nonvolatileSRAM, a flash memory, a type of electronic erasing rewritable read-onlymemory, an erasable programmable read-only memory, a programmableread-only memory, a mask ROM, a floating connecting doors random accessmemory, a nanometer random access memory, a racetrack memory, a variableresistive memory, a programmable metallization cell, or the like, or acombination thereof. The storage device 120 may be a device that storesinformation using magnetic energy, such as a hard disk, a floppy disk, amagnetic tape, a magnetic core memory, a bubble memory, a U disk, aflash memory, or the like, or a combination thereof. The storage device120 may be a device that stores information using optics energy, such asa CD, a DVD, or the like, or a combination thereof. The storage device120 may be a device that stores information using magnetic-opticsenergy, such as a magneto-optical disk. The storage device 120 may storeinformation in, for example, a random storage mode, a serial accessstorage mode, a read-only storage mode, or the like, or a combinationthereof. In some embodiments, the storage device 120 may be anon-permanent memory, a permanent memory, or a combination thereof. Itshould be noted that the above description of storage devices isprovided for the purpose of illustration, and not intended to limit thescope of the present disclosure.

The server 130 may process and/or analyze inputted data (e.g., a medicalimage acquired from the imaging device 110, the storage device 120, orthe external device 140) and generate a result. For example, in aprocess for detecting pulmonary nodules, the server 130 may process andanalyze an inputted medical image related to lungs, and output adetection result about whether the medical image includes a pulmonarynodule. As another example, in a process for diagnosing pulmonarynodules, the server 130 may process and analyze an inputted medicalimage related to lungs, and output a diagnosis result about whether thepulmonary nodule is benign or malignant.

In some embodiments, the server 130 may include a virtualized clusterwork node (e.g., a storm work node). In some embodiments, the sever 130may include one server, or a server group. The server group may becentralized (e.g., a data center), or distributed (e.g., a distributedsystem). The server 130 may include a cloud server, a file server, adatabase server, a FTP server, an application server, a proxy server, amail server, or the like, or a combination thereof. The server 130 maybe local, remote, or a combination thereof. In some embodiments, theserver 130 may access information (e.g., a medical image) stored in thestorage device 120, and/or information in the imaging device 110 (e.g.,a medical image generated by the imaging device 110). In someembodiments, the server 130 may process data in serial by oneworkstation or one server. In some embodiments, the server 130 mayprocess data in parallel. In the parallel processing, the data may besent to one or more servers for processing concurrently by a clusterserver as will be described in connection with FIG. 15.

The external device 140 may input data to the server 130, receive dataoutput from the server 130, and display the outputted data in a form ofdigital, character, image, voice, etc. In some embodiments, the externaldevice 140 may include an input device, an output device, or the like,or a combination thereof. The input device may include a character inputdevice (e.g., a keyboard), an optical reading device (e.g., an opticalmark reader or an optical character reader), a graphic input device(e.g., a mouse, a joystick, or a pen), an image input device (e.g., acamera, a scanner, a fax device, etc.), an analog input device (e.g., ananalog-digital conversion in language recognition system), or the like,or a combination thereof. The output device may include a displaydevice, a printer, a plotter, an image output device, a speech outputdevice, a magnetic recording device, or the like, or a combinationthereof. In some embodiments, the external device 140 may include aninput device and an output device such as a desktop computer, a laptop,a smart phone, a tablet PC, a personal digital assistant (PDA), etc.

In some embodiments, the imaging device 110, the storage device 120, theserver 130, and/or the external device 140 may be connected to and/orcommunicate with each other via a wireless connection, a wiredconnection, or a combination thereof.

It should be noted that the above description of the CAD system 100 ismerely provided for the purposes of illustration, and not intended tolimit the scope of the present disclosure. Many alternatives,modifications, and variations will be apparent to those skilled in theart. The features, structures, methods, and other features of theexemplary embodiments described herein may be combined in various waysto obtain additional and/or alternative exemplary embodiments. Forexample, the storage device 120 may be a cloud computing platformincluding a public cloud, a private cloud, a community and hybrid cloud,etc. However, those variations and modifications do not depart the scopeof the present disclosure.

FIG. 2-A is a block diagram illustrating an example of a computer aideddiagnosis (CAD) system according to some embodiments of the presentdisclosure. As shown, the server 130 may include a processing module210, a storage module 220, an input module 230, and an output module240. The modules or units in the server 130 may be local or remote.

In some embodiments, the input module 230 may receive data sent by theimaging device 110, the storage device 120, the storage module 220, orthe external device 140. The data may include medical data. The medicaldata may include a medical image. The medical image may include anX-rays image, a CT image, a PET image, a MRI image, an ultrasound image,an ECG, an EEG, or the like, or a combination thereof. The medical imagemay be two-dimensional (2D), or three-dimensional (3D). The image formatmay include a joint photographic experts group (JPEG) format, a taggedmedical image file format (TIFF) format, a graphics interchange format(GIF) format, a Kodak FlashPix (FPX) format, a digital imaging andcommunications in medicine (DICOM) format. The data may be input byhandwriting, gesture, image, speech, video, electromagnetic wave, or thelike, or a combination thereof. The data acquired by the input module230 may be stored in the storage module 220, or may be processed and/oranalyzed by the processing module 210.

In some embodiments, the output module 240 may output data processed andanalyzed by the processing module 210. The data may include a detectionand/or diagnosis result, or intermediate data generated during adetection and/or diagnosis processing. For example, in a process fordetecting a pulmonary nodule, the processing module 210 may process andanalyze a medical image. The intermediate data may include asegmentation result of a candidate nodule, feature data of a candidatenodule, etc. The detection result may indicate whether the medical imageincludes a pulmonary nodule. As another example, in a process fordiagnosing a pulmonary nodule, the processing module 210 may process andanalyze a medical image. The intermediate data may include asegmentation result of a candidate nodule, feature data of a candidatenodule, etc. The detection result may indicate whether the pulmonarynodule in the medical image is benign or malignant. The data may be invarious format including a text, an audio, a video, an image, or thelike, or a combination thereof. The outputted data may be transmitted tothe external device 140, or may be transmitted to the storage device 120or the storage module 220 for storing.

In some embodiments, the storage module 220 may store data from theprocessing module 210, the input module 230, and/or the output module240. In some embodiments, the storage module 220 may be integrated inthe system 100, or an external device of the system. The storage module220 may exist in the system substantially, or perform a storage functionby a cloud computing platform.

In some embodiments, the processing module 210 may process data. Theprocessing module 210 may acquire the data from the input module 230 orthe storage module 220. The processing module 210 may transmit theprocessed data to the storage device 120 or the storage module 220 forstoring, or to the output module 240 for outputting the processed data.In some embodiments, the processing module 210 may process the data in aform of storing, classifying, filtering, transforming, calculating,searching, predicting, training, or the like, or a combination thereof.In some embodiments, the processing module 210 may include a centralprocessing unit (CPU), an application-specific integrated circuit(ASIC), an application-specific instruction-set processor (ASIP), agraphics processing unit (GPU), a physics processing unit (PPU), adigital signal processor (DSP), a field programmable gate array (FPGA),a programmable logic device (PLD), a controller, a microcontroller unit,a processor, a microprocessor, an advanced RISC machines processor(ARM), or the like, or a combinations thereof.

It should be noted that the processing module 210 may exist in thesystem 100 substantially, or perform a processing function via a cloudcomputing platform. The cloud computing platform may include a storagecloud platform for storing data, a computing cloud platform forprocessing data, and an integrated cloud computing platform for storingand processing data. The cloud platform configured in the system 100 maybe a public cloud, a private cloud, or a hybrid cloud, or the like, or acombination thereof. For example, according to actual needs, somemedical images received by the CAD system 100 may be calculated and/orstored by the cloud platform. Other medical images may be calculatedand/or stored by a local processing module and/or a database in thesystem.

It should be noted that the system shown in FIG. 2-A may be implementedin various ways. In some embodiments, the system may be implemented by ahardware, a software, or a combination of the software and hardware. Thehardware may be implemented by a special logic. The software may bestored in a storage device and may be implemented by a specifichardware, such as a microprocessor or a special design hardware. Thoseskilled in the art may understand that systems and methods describedabove may be implemented by computer-executable instructions and/orprocessor control codes, provided by, for example a disk, CD or DVD-ROM,a read-only memory (firmware) programmable memory or on a data carriersuch as optical or electrical signal carrier. Systems and modules in thedisclosure may be implemented by a hardware circuit of a programmablehardware device including a super larger scale integrated circle (LSI),a gate array, semiconductor logic chips, transistors, a fieldprogrammable gate array, programmable logic devices, etc., by a softwareexecuted by various processors, or by a combination of the hardwarecircuit and the software.

It should be noted that the above description of the CAD system 100 ismerely provided for the purposes of illustration, and not intended tolimit the scope of the present disclosure. Many alternatives,modifications, and variations will be apparent to those skilled in theart. Modules in the exemplary embodiments described herein may becombined in various ways, or to obtain additional and/or alternativeexemplary embodiments. In some embodiments, the input module 230, theprocessing module 210, the output module 240, and the storage module 220may exist in different modules of one system, or one module may performfunctions of two or more modules. For example, the processing module 210and the storage module 220 may be two modules, or integrated into onesingle module for processing and storing. As another example, eachmodule may share one storage module, or each module may include astorage module respectively. However, those variations and modificationsdo not depart the scope of the present disclosure.

FIG. 2-B is a flowchart illustrating an exemplary process for a computeraided diagnosis (CAD) according to some embodiments of the presentdisclosure. In some embodiments, the process for the computer aideddiagnosis (CAD) may be performed by the server 130.

In 252, original data may be acquired from the imaging device 110, theexternal device 140, the storage device 120, and/or the storage module220. In some embodiments, the original data may include medical data.The medical data may include a medical image. The medical image mayinclude an X-rays image, a CT image, a PET image, an MRI image, anultrasound image, an ECG, an EEG, or the like, or a combination thereof.The medical image may be two-dimensional (2D), or three-dimensional(3D). The format of the medical image may include a joint photographicexperts group (JPEG), a tagged medical image file format (TIFF),graphics interchange format (GIF), a Kodak Flash Pix (FPX), a digitalimaging communications in medicine (DICOM), etc. The form of theoriginal data may include a text, an audio, a video, an image, or thelike, or a combination thereof.

In 254, the original data may be processed. In some embodiments, theprocessing of the original data may include storing, classifying,filtering, transforming, computing, searching, predicting, training, orthe like, or a combination thereof.

For illustration purposes, a prediction model and a machine learningtechnique applied in some embodiments of processing the original datamay be illustrated below. In some embodiments, the prediction model maybe qualitative or quantitative. For example, the quantitative predictionmodel may include applying a time series prediction technique or acausal analysis technique. The time series prediction technique mayinclude an average smoothing technique, a trend extrapolation technique,a seasonal conversion prediction technique, a Markov predictiontechnique, or the like, or a combination thereof. The causal analysistechnique may include a unitary regression technique, a multipleregression technique, an input-output analysis technique, or the like,or a combination thereof. In some embodiments, the prediction model mayinclude a weighted arithmetic average model, a trend average predictionmodel, an exponential smoothing model, an average growth rate model, aunitary linear regression model, a high and low point mode, or the like,or a combination thereof.

In some embodiments, equations, algorithms and/or models applied inprocessing data may be optimized based on the machine learningtechnique. The machine learning technique according to a learningmechanism may include a supervised learning, an unsupervised learning, asemi-supervised learning, reinforcement learning, or the like, or acombination thereof. In some embodiments, the machine learning techniquemay include a regression algorithm, a case learning algorithm, a formallearning algorithm, a decision tree learning algorithm, a Bayesianlearning algorithm, a kernel learning algorithm, a clustering algorithm,an association rules learning algorithm, a neural network learningalgorithm, a deep learning algorithm, a dimension reduction algorithm,etc. The regression algorithm may include a logistic regressionalgorithm, a stepwise regression algorithm, a multivariate adaptiveregression splines algorithm, a locally estimated scatterplot smoothingalgorithm, etc. The case learning algorithm may include a k-nearestneighbor algorithm, a learning vector quantization algorithm, aself-organizing map algorithm, etc. The formal learning algorithm mayinclude a ridge regression algorithm, a least absolute shrinkage andselection operator (LAASSO) algorithm, an elastic net algorithm, etc.The decision tree learning algorithm may include a classification andregression tree algorithm, an iterative dichotomiser 3 (ID3) algorithm,a C4.5 algorithm, a chi-squared automatic interaction detection (CHAID)algorithm, a decision stump algorithm, a random forest algorithm, a marsalgorithm, a gradient boosting machine (GBM) algorithm, etc. TheBayesian learning algorithm may include a naive Bayesian algorithm, anaveraged one-dependence estimators algorithm, a Bayesian belief network(BBN) algorithm, etc. The kernel learning algorithm may include asupport vector machine algorithm, a linear discriminate analysisalgorithm, etc. The neural network learning algorithm may include aperceptron neural network algorithm, a back propagation algorithm, aHopfield network algorithm, a self-organizing map (SOM) algorithm, alearning vector quantization algorithm, etc. The deep learning algorithmmay include a restricted Boltzmann machine algorithm, a deep beliefnetworks (DBN) algorithm, a convolutional neural network algorithm, astacked auto-encoders algorithm, etc. The dimension reduction algorithmmay include a principle component analysis algorithm, a partial leastsquare regression algorithm, a Sammon mapping algorithm, amulti-dimensional scaling algorithm, a projection pursuit algorithm,etc.

In 256, a processing result may be output. In some embodiments, theprocessing result may be transmitted to the external device 140, orstored in the storage device 120 or the storage module 220. In someembodiments, the form of the processing result may include a text, animage, an audio, a video, or the like, or a combination thereof.

It should be noted that the flowchart depicted above is provided for thepurposes of the illustration, and not intended to limit the scope of thepresent disclosure. For persons having ordinary skills in the art,various variations and modification may be conducted under the teachingof the present disclosure. For example, one or more operations may beomitted or added. For example, the original data may be preprocessed.The preprocessing may include removing fuzzy data by a data cleaning, adata integration, a data transformation, a data specification, etc. Insome embodiments, the fuzzy data may be removed based on a discriminanttechnique, an elimination technique, an average technique, a smoothingtechnique, a proportion technique, a moving average technique, anexponential smoothing technique, a difference technique, etc. As anotherexample, the process for the CAD may include a step for storing data.However, those variations and modifications may not depart the scope ofthe present disclosure.

FIG. 3 is a block diagram illustrating an example of a processing module210 in the server 130 according to some embodiments of the presentdisclosure. As shown, the processing module 210 may include an objectdetermination sub-module 211, a training sub-module 213, and aclassification sub-module 215. The object determination sub-module 211may determine a candidate target (e.g., a suspected pulmonary nodule).The training sub-module 213 may train a classifier based on thedetermined candidate target. The classification sub-module 215 mayclassify the candidate target. For example, the classificationsub-module 215 may determine whether a suspected pulmonary nodule is atrue pulmonary nodule, or determine whether a true pulmonary nodule isbenign or malignant. In some embodiments, besides the sub-modulesdescribed above, the processing module 210 may include one or more othermodules or units. The modules 310-380 may be connected to or communicatewith each other via a wired connection or a wireless connection.

The object determination sub-module 211 may include an original dataacquisition unit 310, a region of interest (ROI) determination unit 320,a preprocessing unit 330, and a candidate target determination unit 340.

The original data acquisition unit 310 may obtain original data. In someembodiments, the original data may include medical data. The medicaldata may include a medical image. The medical image may include anX-rays image, a CT image, a PET image, a MRI image, an ultrasound image,an ECG, an EEG, or the like, or a combination thereof. The medical imagemay be two-dimensional (2D), or three-dimensional (3D). The image formatmay include a joint photographic experts group (JPEG) format, a taggedmedical image file format (TIFF) format, a graphics interchange format(GIF) format, a Kodak FlashPix (FPX) format, a digital imaging andcommunications in medicine (DICOM) format. In some embodiments, theoriginal data acquisition unit 310 may acquire the original data fromthe imaging device 110, the external device 140, the storage device 120,and/or the storage module 220. In some embodiments, the original datamay be acquired in real time, or non-real time. In some embodiments, theacquired original data may be stored in the storage device 120, thestorage module 220, and/or any other storage device integrated in orindependent from the system as described elsewhere in the disclosure. Insome embodiments, the original data acquired by the original dataacquisition unit 310 may be transmitted to other modules, sub-modules,units and/or sub-units for further processing. For example, the originaldata acquisition unit 310 may transmit the original data to the ROIdetermination unit 320 to determine a ROI. As another example, theoriginal data may be transmitted to the preprocessing unit 330 forpreprocessing. As another example, the original data acquisition unit310 may transmit the original data to the candidate target determinationunit 340 to determine a candidate target.

The ROI determination unit 320 may determine a ROI based on the originaldata. In some embodiments, the ROI may include a candidate target. Forexample, in a process for detecting pulmonary nodules in a medicalimage, the ROI may include a pulmonary parenchyma region in the medicalimage. The lungs may include pulmonary parenchyma and pulmonarymesenchyme. The pulmonary parenchyma may include different levelsbranches of bronchi and alveoli at the terminal of the bronchi. Thepulmonary mesenchyme may include connective tissues, blood vessels,lymphatic vessels, nerves, etc. In some embodiments, data processed bythe ROI determination unit 320 may be transmitted to other modules,sub-modules, units, and/or sub-units for further processing. Forexample, the ROI determination unit 320 may transmit the processed datato the preprocessing unit 330 for preprocessing. As another example, theROI determination unit 320 may transmit the processed data to thecandidate target determination unit 340 to determine a candidate target.

The preprocessing unit 330 may preprocess the original data and/or theROI. In some embodiments, the preprocessing may include preliminarypositioning, enhancement processing, interpolation processing,morphology processing, denoising processing, or the like, or acombination thereof. The preliminary positioning may be performed todetermine a rough region of a candidate target in the original data(e.g., an original medical image) or the ROI to simplify the process ofdetermining the candidate target and play a basic role in the process ofdetermining the candidate target. The preliminary positioning may beperformed automatically, semi-automatically, manually, etc. Theenhancement processing may be performed to highlight a structure orregion in the original data (e.g., an original medical image) or an ROI.The enhancement processing may include a Hessian dot-enhancement, aHessian line-enhancement, or the like, or a combination thereof. Theinterpolation processing may be performed to balance voxel sizes in theoriginal data (e.g., an original medical image) or an ROI. Themorphology processing may be performed to analyze and identify a targetby processing a shape of a structure in the original data (e.g., anoriginal medical image) or an ROI based on an element with a specificmorphological structure. The morphology processing may include anexpansion operation, a corrosion operation, an open operation, a closedoperation, or the like, or a combination thereof. The denoisingoperation may be performed to remove noise caused by machines and/orobject motions in the original data (e.g., an original medical image) oran ROI. In some embodiments, the preprocessed data may be transmitted toother modules, sub-modules, units, and/or sub-units for furtherprocessing. For example, the preprocessed data may be transmitted to thecandidate target determination unit 340 to determine a candidate target.

The candidate determination unit 340 may determine a candidate target.In a process for detecting pulmonary nodules, the candidate target maybe suspected pulmonary nodules. In a process for detecting breast lumps,the candidate target may be suspected breast lumps. In some embodiments,data processed by the candidate target determination unit 340 may betransmitted to any other module, sub-module, unit and/or sub-unit forfurther processing. For example, data processed by the candidatedetermination unit 340 may be transmitted to the first featureextraction unit 351 and/or the second feature extraction unit 353 toextract features.

The training sub-module 213 may include a first feature extraction unit351, a feature selection unit 360, and a training unit 370.

The first feature extraction unit 351 may extract features from acandidate target in a training process. In some embodiments, thefeatures may be used to distinguish a target from other targets. Thefeatures may be data exacted through processing and/or measuring thetarget. For example, in a process for detecting pulmonary nodules, thecandidate target may be pulmonary nodules. The features may include agray value related feature, a morphological feature, a texture feature,a serial section feature, or the like, or a combination thereof. Thegray value related feature may include a mean gray value, a gray valuevariance, etc. The mean gray value may denote an average gray value ofthe gray values of all pixels in a suspected pulmonary nodule. The grayvalue variance may denote the intensity of change in gray values ofpixels in a suspected pulmonary nodule.

In a 2D image, the morphological feature may include an area, aperimeter, a centroid, a diameter, a curvature, an eccentricity, aroundness, a compactness, a Fourier descriptor, a shape momentdescriptor, or the like, or a combination thereof. The area may denote apixel number in a candidate target region. The perimeter may denote apixel number on the boundary of a candidate target. The centroid maydenote an abscissa average value and an ordinate average valuecorresponding to all the pixels in a candidate target region. Thediameter may denote a distance between any two pixels on the boundary ofa candidate target. The eccentricity may denote an extent in which acandidate target is close to a circle. The roundness may measure aroundness of a candidate target. The compactness may denote the boundarysmoothness of a candidate target. The Fourier descriptor may be used todetermine whether a target (e.g., an isolated pulmonary nodule) has burrcharacterization. The shape moment descriptor may denote the boundaryshape of a target (e.g., a pulmonary nodule).

In a 3D image, the feature may include a volume, a perimeter, acentroid, a diameter, a curvature, an eccentricity, a sphericity, acompactness, a Fourier descriptor, a shape moment descriptor, or thelike, or a combination thereof. The volume may denote the number ofvoxels in a candidate target region. The sphericity may measure thesphericity of a candidate target. In a 2D and/or 3D image, the texturefeature may include a statistic magnitude, a boundary clarity factor, afrequency domain parameter, etc. The statistic magnitude may be used toassess the spatial correlation between the change in gray values andpixels or voxels in an image. The boundary clarity factor may denote aboundary clarity of a target (e.g., a pulmonary nodule). The statisticmagnitude and the boundary clarity factor may denote a suspected target(e.g., a pulmonary nodule) in the spatial domain. The frequency domainparameter may denote a suspected target (e.g., a pulmonary nodule) inthe frequency domain. In some embodiments, the first feature extractionunit 351 may extract all features of a candidate target for generating auniversal set of features. A reference subset of features may beextracted from the universal set of features.

The feature selection unit 360 may generate a reference subset offeatures by selecting features from the extracted features. Thereference subset of features may be used to train a classifier. In someembodiments, the extracted features may be transmitted to any othermodule, sub-module, unit, and/or sub-unit for further processing. Forexample, the extracted features may be transmitted to the featureselection unit 360 to select features.

The training unit 370 may train a classifier. In some embodiments, aprocess for training a classifier may include deriving a classificationfunction or generating a classification model based on acquired data.The classification function or the classification model may also bereferred to as a classifier. In some embodiments, the classifier trainedby the training unit 370 may be transmitted to any other module,sub-module, unit, and/or sub-unit for further processing. For example,the training unit 370 may transmit the classifier to the classificationunit 380 for classification.

The classification sub-module 215 may include a classifier acquisitionunit 375, a second feature extraction unit 353, and a classificationunit 380.

The classifier acquisition unit 375 may acquire a classifier. Theclassifier may include a reference subset of features.

The second feature extraction unit 353 may extract features from acandidate target. In some embodiments, the second feature extractionunit 353 may extract the features from the candidate target based on thereference subset of features.

The classification unit 380 may generate a classification result byclassifying a candidate target according to the extracted features fromthe candidate target. In some embodiments, a classification process mayinclude performing data prediction by mapping features of a candidatetarget to a specific category based on a classification model orfunction. A classifier may correspond to a classification technique. Insome embodiments, the classification technique may include a supervisedtechnique. The supervised technique may include identifying a pendingsample according to a specific rule and element features of the pendingsample, and classifying the element features of the pending sample intoa category of training samples based on similar features to the pendingsample. The supervised technique may include a linear discriminantalgorithm, an artificial neural network algorithm, a Bayesclassification algorithm, a support vector machine (SVM) algorithm, adecision tree algorithm, a logistic regression algorithm, or the like,or a combination thereof.

For example, in a process for detecting a pulmonary nodule, aclassification result may include whether the suspected pulmonary noduleis a true pulmonary nodule. As another example, in a process fordiagnosing a pulmonary nodule, a classification result may includewhether the pulmonary nodule is benign or malignant. In someembodiments, the classification result may be expressed as aprobability. For example, in a process for detecting a pulmonary nodule,a classification result may include a probability of the suspectedpulmonary nodule to be a true pulmonary nodule. As another example, in aprocess for diagnosing a pulmonary nodule, a classification result mayinclude a probability of the pulmonary nodule to be malignant.

In some embodiments, data generated by the object determinationsub-module 211, the training sub-module 213, and/or the classificationsub-module 215 may be transmitted to any other module, unit, and/orsub-unit for further processing. The data may be stored in the storagedevice 120 and/or the storage module 20, or may be output to theexternal device 140 by the output module 240.

In some embodiments, the processing module 210 may include one or morestorage modules (not shown in FIG. 3). The storage modules may storeinformation and intermediate data extracted, determined, and/orgenerated by each sub-module and/or unit. In some embodiments, thesub-modules 211, 213, and 215 in the processing module 210 may includeone or more storage units (not shown in FIG. 3) for storing informationand intermediate data.

In some embodiments, the sub-modules 211, 213, and 215 in the processingmodule 210 may perform an operation or processing based on a logicoperation (e.g., AND, OR, NOT operation, etc.), a numerical operation,or the like, or a combination thereof. The units 310-380 in theprocessing module 210 may include one or more processors. The processorsmay include any general processor. For example, the processors mayinclude a programmable logic device (PLD), an application specificintegrated circuit (ASIC), a microprocessor, a system chip (SoC), adigital signal processor (DSP), etc. In some embodiments, two or moreunits of the units 310-380 may be integrated on one, two, or morehardware independently. It should be appreciated that the sub-modules211, 213, and 215 in the processing module 210 may be implemented invarious forms. For example, the system may be implemented by a hardware,a software, or a combination thereof. The hardware may include a superLSI, a gate array, a semiconductor logic chip, a transistor, a fieldprogrammable gate array, a programmable logic device, or the like, or acombination thereof. The software may be implemented by variousprocessors.

In some embodiments, the training sub-module 213 may be an off-linemodule that may perform the training of a classifier offline.

It should be noted that the description for the processing module 210above is provided for the purposes of the illustration, and not intendedto limit the scope of the present disclosure. Many alternatives,modifications, and variations to the processing module 210 will beapparent to those skilled in the art. The features, structures, methods,and other features of the exemplary embodiments described herein may becombined in various ways to obtain additional and/or alternativeexemplary embodiments. For example, the ROI determination unit 320and/or the preprocessing unit 330 may be omitted. As another example,the first feature extraction unit 351 and the second feature extractionunit 353 may be integrated into one single unit for extracting features.However, those variations and modifications do not depart the scope ofthe present disclosure.

FIG. 4-A is a flowchart illustrating an exemplary process fordetermining a candidate target by a processing module 210 according tosome embodiments of the present disclosure. In 412, original data may beacquired. Operation 412 may be performed by the original dataacquisition unit 310. The original data may include medical data. Insome embodiments, the medical data may include a medical image. Themedical image may include an X-rays image, a CT image, a PET image, aMRI image, an ultrasound image, an ECG, an EEG, or the like, or acombination thereof. The medical image may be two-dimensional (2D), orthree-dimensional (3D). The image format may include a jointphotographic experts group (JPEG) format, a tagged medical image fileformat (TIFF) format, a graphics interchange format (GIF) format, aKodak FlashPix (FPX) format, a digital imaging and communications inmedicine (DICOM) format. In some embodiments, the original data may beacquired from the imaging device 110, the storage device 120, thestorage module 220, or the external device 140.

In 414, a first region of interest (ROI) may be determined based on theoriginal data. Operation 414 may be performed by the ROI determinationunit 320. In some embodiments, the range for determining a candidatetarget may be shrank before determining the candidate target to reducethe difficulty and complexity for determining the candidate target. Insome embodiments, the ROI may include a candidate target. For example,in a process for detecting a pulmonary nodule, the ROI may include apulmonary parenchyma region in a medical image. The determination of thepulmonary parenchyma region may include extracting information in thepulmonary parenchyma region and remove information out of the pulmonaryparenchyma region, such as lung cavity, clothes, fat, or otherinterference factors. In some embodiments, the ROI may be determinedbased on a thresholding algorithm, a region growing algorithm, an edgedetection algorithm, a morphology processing algorithm, a multi-scaleanalysis algorithm, a pattern analysis algorithm, a clusteringalgorithm, or the like, or a combination thereof. In some embodiments,the image generated by determining the ROI may be a mask image (as shownin FIG. 17-B) if the original data is a medical image. In someembodiments, the generation of the mask image may include multiplying apredetermined ROI mask image by the medical image. Then the pixel orvoxel value corresponding to a pixel or voxel in the ROI may beconstant, and the pixel or voxel value corresponding to a pixel or voxelout of the ROI may be 0. As used herein, the pixel or voxel value maydenote a color value of a pixel or voxel in an image. For a monochromeimage, the pixel or voxel value may denote a gray value of a pixel orvoxel in an image.

In 416, a candidate target or a second region of interest (ROI) may bedetermined by preprocessing the first ROI. Operation 416 may beperformed by the preprocessing unit 330. In some embodiments, thepreprocessing may include preliminary positioning, enhancementprocessing, interpolation processing, morphology processing, etc. Thepreliminary positioning may be performed to determine an approximateregion of a candidate target in the first ROI to simplify the process ofdetermining the candidate target and play a basic role in the processfor determining the candidate target. The preliminary positioning may beperformed automatically, semi-automatically, manually, etc. Theenhancement processing may be performed to highlight a structure orregion in the ROI. The enhancement processing may include a Hessiandot-enhancement, a Hessian line-enhancement, or the like, or acombination thereof. The interpolation processing may be performed tobalance voxels sizes in the ROI. The morphology processing may beperformed to analyze and identify a target by processing a shape of astructure in the ROI based on an element with a specific morphologicalstructure. The morphology processing may include an expansion operation,a corrosion operation, an open operation, a closed operation, or thelike, or a combination thereof.

In 418, a candidate target may be determined by segmenting thepreprocessed data based on one or more segmentation models. Operation418 may be performed by the candidate target determination unit 340. Insome embodiments, the segmentation may include dividing an image intoseveral portions based on a specific uniformity (or consistency)principle, such that each portion may satisfy the specific consistency.In some embodiments, the segmentation may also refer to a classificationof pixels or voxels in an image. In some embodiments, a segmentationmodel may correspond to a segmentation technique. In some embodiments,the segmentation model may include a threshold segmentation model, aregion growing segmentation model, a watershed segmentation model, astatistical segmentation model, a morphological segmentation model, orthe like, or a combination thereof. The threshold segmentation model mayinclude classifying pixels or voxels in an image into different types bysetting different thresholds. The threshold segmentation model mayinclude a single threshold segmentation model and/or a multi-thresholdsegmentation model according to the number of thresholds. The thresholdsegmentation model may include an iterative threshold segmentationmodel, a histogram segmentation model, or the Otsu segmentationalgorithm. The region growing segmentation model may start from agrowing point (e.g., a single pixel or voxel, or a region) and mergeadjacent pixels or voxels including one or more similar features such asgray value, texture, color, etc. to the growing point into a sameregion. The region growing segmentation model may include an iterativeprocess of adding a new pixel or voxel in to the growing region. Theiterative process may end until there is no near point to be merged intothe growing region. The watershed segmentation model may include aniterative labeling process. The gray level of each pixel or voxel may beranked from low to high, then in a submergence process from low to high,each local minimum value may be determined and labeled by applying afirst-in first-out (FIFO) structure in an h order domain of influence.The morphological segmentation model may include a Hessiandot-enhancement model, a Hessian-line enhancement model, a multi-scaleGaussian template matching model, a multi-scale morphological filteringmodel, etc. The Hessian dot-enhancement model may be applied to enhance,for example, a dot and/or an approximate dot in an image. The Hessiandot-enhancement model may be applied to enhance a line-type graph in animage. The multi-scale Gaussian template matching model may be appliedto segment an image based on the morphology of a candidate target. Forexample, the multi-scale Gaussian template matching model may be appliedto segment pulmonary nodules based on the approximate round morphologyof the pulmonary nodules. The multi-scale morphological filtering modelmay apply various mathematical morphology techniques to perform afiltering operation on an image. An edge-based segmentation model mayinclude a level set algorithm. The statistical segmentation model mayinclude a variational expectation maximization, a K-means model, a FuzzyC-means model, or the like, or a combination thereof.

In some embodiments, the preprocessed data may be segmented based on onesegmentation model, or multi-segmentation models for determining acandidate target. The emphasis of different segmentation models may bedifferent when the preprocessed data is segmented based on themulti-segmentation models. For example, the morphological segmentationmodel (e.g., a morphology-based region growing segmentation model) maysegment an image based on gray values of pixels or voxels to obtain asegmentation result with a fixed morphology. The statisticalsegmentation model (e.g., a VEM model) may segment an image based on thegray value distribution of pixels or voxels. The candidate targetdetermined by segmenting a same image region based on different modelsmay be different. The combination of different models may becomplementary. For example, a pulmonary nodule may be a solid nodule, amixed ground glass nodule, a ground glass nodule, etc. The pulmonarynodule may not be identified based on the morphological segmentationmodel effectively. The combination of the morphological segmentationmodel and the statistical segmentation model may identify the pulmonarynodules effectively. For example, multiple candidate targets may bedetermined by processing a same region based on different segmentationmodels. The multiple candidate targets may be complementary by eachother. The universal set of features of the candidate targets may beexacted from the multiple candidate targets. The universal set offeatures may include more effective feature information and enlarge therange of the universal set of features.

In a process for detecting a pulmonary nodule, the candidate target mayinclude a suspected pulmonary nodule. In a process for detecting abreast lump, the candidate target may include a suspected breast lump.

It should be noted that the flowchart for determining a candidate targetdepicted above is provided for the purposes of the illustration, and notintended to limit the scope of the present disclosure. For personshaving ordinary skills in the art, various variations and modificationmay be conducted under the teaching of the present disclosure. However,those variations and modifications may not depart the scope of thepresent disclosure. For example, a denoising operation may be performedbetween operation 412 and operation 414 for removing noise in theoriginal data. As another example, operation 414 or operation 416 may beomitted. In some embodiments, an enhancement operation may be addedafter operation 418 for enhancing the determined candidate target. Insome embodiments, other selection conditions may be added between twooperations. For example, a result generated by any one operation may bestored or be backed up.

FIG. 4-B is a flowchart illustrating an exemplary process for training aclassifier by a processing module 210 according to some embodiments ofthe present disclosure.

In 422, a target to be processed may be acquired. Operation 412 may beperformed by the first feature extraction unit 351. In some embodiments,the target to be processed may include a candidate target, a truetarget, original data, or preprocessed data. The preprocessed data mayinclude an interpolation image, a Hessian dot-enhanced image, a Hessianline-enhancement image, a morphology image, or the like, or acombination thereof. For example, the candidate target may be asuspected pulmonary nodule in a process for training a classifier fordetecting a pulmonary nodule. The candidate target may be a truepulmonary nodule in a process for training a classifier for diagnosing apulmonary nodule.

In 424, a universal set of features may be determined by extractingfeatures from the target to be processed. Operation 424 may be performedby the first feature extraction unit 351. In some embodiments, thefeatures may be extracted for classifying a target from other targetsthrough processing and/or measuring the target. For example, in aprocess for detecting a pulmonary nodule, the candidate target may be asuspected pulmonary nodule. The features may include a gray valuerelated feature, a morphological feature, a texture feature, a serialslice feature, or the like, or a combination thereof. The gray valuerelated feature may include a mean gray value, a gray value variance, orthe like, or a combination thereof. The morphological feature in a 2Dimage may include may include an area, a perimeter, a centroid, adiameter, a curvature, an eccentricity, a roundness, a compactness, aFourier descriptor, a shape moment descriptor, or the like, or acombination thereof. The morphological feature in a 3D image may includea volume, a perimeter, a centroid, a diameter, a curvature, aneccentricity, a sphericity, a compactness, a Fourier descriptor, a shapemoment descriptor, or the like, or a combination thereof. In someembodiments, the universal set of features may include all featuresassociated with the target. In some embodiments, the universal set offeatures may be extracted based on original data, an ROI, orpreprocessed data. In some embodiments, the universal set of featuresmay include features extracted from the candidate target, featuresextracted from the original data, features extracted from thepreprocessed data, or the like, or a combination thereof. In someembodiments, the universal set of features may include non-image data.The non-image data may include medical history data, clinical trialresults, symptoms, physiological data, pathological data, genetic data,or the like, or a combination thereof. For example, the non-image datafor detecting a pulmonary nodule may include age, gender, smokinghistory, cancer history, family history, occupation exposure, leisureexposure, previous lung disease, current lung disease, previous thoracicsurgery, the number of satellite lesions around diagnosed lesions, lymphnode size, suspicious lesions, lesions location in the lung, or thelike, or a combination thereof.

In 426, a reference subset of features may be determined by selectingfeatures from the universal set of features. Operation 426 may beperformed by the first feature extraction unit 351. In some embodiments,the purpose of selecting features may be to improve the accuracy of aclassifier. In some embodiments, in the process of selecting features,for different purposes, features in the universal set of features thatare important for classifying the target may be selected as features inthe reference subset of features. For example, in a process fordetecting a pulmonary nodule, the classification may determine whether asuspected pulmonary nodule is a true pulmonary nodule. Features (e.g., amean gray value, a gray variance, etc.) important for determining thetrue pulmonary nodule may be selected, and features (e.g., a frequencydomain parameters feature) without much impact on determination of thetrue pulmonary nodule may not be selected. In a process for diagnosing apulmonary nodule, the classification may determine whether the pulmonarynodule is benign or malignant. Features (e.g., a boundary clarityfactor) important for determining a benign or malignant pulmonary nodulemay be selected, and features (e.g., a circularity, a sphericity, etc.)without much impact on the determination of the benign or malignantpulmonary nodule may not be selected. The features may be selected basedon an exhaustive algorithm, a heuristic algorithm, a random algorithm,or the like, or a combination thereof. The random algorithm may includea genetic algorithm, a particle swarm algorithm, a simulated annealingalgorithm, etc. The exhaustive algorithm may include a breadth firstsearch algorithm, a depth first search algorithm, etc. The heuristicalgorithm may include a decision tree algorithm, a Relief algorithm, aforward (backward) algorithm, etc.

In 428, a classifier may be determined by training a classifier based onthe reference subset of features. Operation 428 may be performed by thetraining unit 370. In some embodiments, the classifier training mayinclude deriving a classification function or establish a classificationmodel based on existing data. The classification function or theclassification model may also be referred to as a classifier. In someembodiments, the classifier training may include determining a trainingsample including a positive sample and a negative sample, extractingfeatures from the training sample based on the reference subset offeatures, performing a training algorithm on the extracted features fromthe training sample for generating a classification function or model.The classification function or model may be also referred to as aclassifier. In some embodiments, the positive sample may include thetarget to be processed and the negative sample may not include thetarget to be processed. For example, the positive sample may include apulmonary nodule and the negative sample may not include the pulmonarynodule for training a classifier used to determine a true pulmonarynodule. As another example, the positive sample may include a benignpulmonary nodule (or a malignant pulmonary nodule) and the negativesample may not include a benign pulmonary nodule (or a malignantpulmonary nodule) if the classifier is used to determine a benign ormalignant pulmonary nodule. In some embodiments, the classifier trainedbased on the reference subset of features may include the referencesubset of features.

In some embodiments, if the target to be processed acquired in 422 is acandidate target, the classifier determined in 428 may include adetection classifier. The detection classifier may determine whether thecandidate target is a true target. If the target to be processedacquired in 422 is a true target, the classifier determined in 428 mayinclude a diagnosis classifier. The diagnosing classifier may determinethe property of the true target (e.g., benign or malignant). Forexample, if the target to be processed acquired in 422 is a suspectedpulmonary nodule, the classifier determined in 428 may include adetection classifier. The detection classifier may determine whether thesuspected pulmonary nodule is a true pulmonary nodule. If the target tobe processed acquired in 422 is a true pulmonary nodule, the classifierdetermined in 428 may include a diagnosing classifier. The diagnosingclassifier may determine whether the true pulmonary nodule is benign ormalignant.

FIG. 4-C is a flowchart illustrating an exemplary process for performingclassification based on a classifier by a processing module 210according to some embodiments of the present disclosure. In 432, atarget to be processed may be acquired. Operation 432 may be performedby the second feature extraction unit 353. In some embodiments, thetarget to be processed may include a candidate target, a true target,original data, or preprocessed data. The preprocessed data may includean interpolation image, a Hessian dot-enhanced image, a Hessianline-enhancement image, a morphology image, or the like, or acombination thereof. For example, in a process for detecting a pulmonarynodule, the candidate target may be a suspected pulmonary nodule. In aprocess for diagnosing a pulmonary nodule, the candidate target may be atrue pulmonary nodule.

In 434, a classifier including a reference subset of features may beacquired. Operation 434 may be performed by the classifier acquisitionunit 375. In some embodiments, the classifier may be determined asdescribed in connection with FIG. 4-B. In some embodiments, thereference subset of features may include two classification indicators.The first classification indicator may include a feature type determinedin a process for selecting features. The second classification indicatormay include a category indicator for classifying features. For example,in a process for classifying features, a quantized value correspondingto a specific feature may be compared with the category indicator todetermine the category of the specific feature.

In 436, feature data may be determined by extracting features from thetarget to be processed based on the reference subset of features.Operation 436 may be performed by the second feature extraction unit353.

In 438, a classification result may be determined by classifying thetarget to be processed based on the classifier and the feature data.Operation 438 may be performed by the classification unit 380. In someembodiments, a classifying processing may include performing dataprediction by mapping features of a candidate target to a specificcategory based on a classification model or function. In someembodiments, the classification of the target to be processed mayinclude comparing a quantized value corresponding to the feature data inthe reference subset of features with a category indicator, determiningthe type of the feature data, and determining the type of the target tobe processed. In some embodiments, a classifier may correspond to aclassification technique. In some embodiments, the classificationtechnique may include a supervised technique. The supervised techniquemay include identifying a pending sample according to a specific ruleand element features of the pending sample, and classifying the elementfeatures of the pending sample into a category of training samples basedon similar features to the pending sample. The supervised technique mayinclude a linear discriminant algorithm, an artificial neural networkalgorithm, a Bayes classification algorithm, a support vector machine(SVM) algorithm, a decision tree algorithm, a logistic regressionalgorithm, or the like, or a combination thereof. For example, in aprocess for detecting a pulmonary nodule, a classification result mayinclude whether the suspected pulmonary nodule is a true pulmonarynodule. As another example, in a process for diagnosing a pulmonarynodule, a classification result may include whether the pulmonary noduleis benign or malignant. In some embodiments, the classification resultmay be expressed as a probability. For example, in a process fordetecting a pulmonary nodule, a classification result may include aprobability of the suspected pulmonary nodule to be a true pulmonarynodule. As another example, in a process for diagnosing a pulmonarynodule, a classification result may include a probability of thepulmonary nodule to be malignant.

In some embodiments, if the target to be processed acquired in 432 is acandidate target, the classification process may be referred to as adetection process. The detection process may be performed to determinewhether a candidate target is a true target. If the target to beprocessed acquired in 432 is a true target, the classification processmay be referred to as a diagnosing process. The diagnosis process may beperformed to determine a property of the true target. For example, ifthe target to be processed acquired in 432 is a suspected pulmonarynodule, the classification process may be referred to as a detectingprocess. The detecting process may be performed to determine whether thesuspected pulmonary nodule is a pulmonary nodule. As another example, ifthe target to be processed acquired in 432 is a pulmonary nodule, theclassification process may be referred to as a diagnosis process. Thediagnosis process may be performed to determine whether the pulmonarynodule is benign or malignant.

In some embodiments, the process described in FIG. 4-B may be performedto train a detection classifier or a diagnosis classifier. For example,if the target to be processed acquired in 422 is a candidate target, theprocess described in FIG. 4-B may be performed to train a detectionclassifier. The candidate target may be determined according to theprocess described in connection with FIG. 4-A. As another example, ifthe target to be processed acquired in 432 is a candidate target, theprocess described in FIG. 4-C may be performed to detect the candidatetarget. The candidate target may be determined according to the processdescribed in connection with FIG. 4-A.

In some embodiments, the process described in FIG. 4-B may be combinedwith the process described in FIG. 4-C. For example, a detectionclassifier determined according to the process described in FIG. 4-B maybe used in a detecting process described in FIG. 4-C. As anotherexample, a diagnosing classifier determined according to the processdescribed in FIG. 4-B may be used in a diagnosis process described inFIG. 4-C. As still another example, a true target determined accordingto the process described in FIG. 4-C may be used to train a diagnosingclassifier described in connection with FIG. 4-B.

In some embodiments, a detecting process and a diagnosis processdescribed in FIG. 4-C may be combined. For example, operations 434-438may be performed on a candidate target to determine a true target, andthen operations 434-438 may be performed on the true target to determinea property of the true target. For example, operations 434-438 may beperformed on a suspected pulmonary nodule to determine a pulmonarynodule, and then operations 434-438 may be performed on the pulmonarynodule to determine whether the pulmonary nodule is benign or malignant.

In some embodiments, the process described in FIG. 4-A, the processdescribed in FIG. 4-B, and the process described in FIG. 4-C may becombined. For example, a candidate target may be acquired according tothe process described in FIG. 4-A. A detection classifier may be trainedbased on the candidate target according to operations 424-428 describedin FIG. 4-B. The candidate target may be determined to be a true targetor not based on a detection classifier according to operations 434-438described in FIG. 4-C. The detection classifier acquired in 434 may betrained according to the process described in FIG. 4-B. A diagnosingclassifier may be trained based on the true target according tooperations 424-428 described in FIG. 4-B. The property of the truetarget may be determined based on the diagnosing classifier according tooperations 434-438 described in FIG. 4-C. The diagnosing classifieracquired in 434 may be trained according to the process described inFIG. 4-B.

FIG. 5 is a block diagram illustrating an example of a candidate targetdetermination unit 340 according to some embodiments of the presentdisclosure. As shown, the candidate target determination unit 340 mayinclude a surrounding region determination sub-unit 510, a substanceregion determination sub-unit 520, and a combination sub-unit 530. Insome embodiments, the surrounding region determination sub-unit 510 maydetermine a region around a candidate target. The substance regiondetermination sub-unit 520 may determine a substance region of acandidate target. In some embodiments, a candidate target may include asubstance region and a surrounding region in morphology. The substanceregion may represent a main body of the candidate target, and thesurrounding region may represent an edge of the candidate target. Forexample, a pulmonary nodule may be one or more circular (spherical orapproximatively spherical in a three-dimensional image) dense shadows inthe lung parenchyma with a diameter less than 3 cm. The pulmonary nodulerepresented in a medical image may include a definite edge, such as asmoothing edge, a lobulated edge, thorns, spines, or the like, or acombination thereof. The substance region of a pulmonary nodule may be acircular or quasi-circular region (spherical or approximativelyspherical in a three-dimensional image) in the pulmonary nodule. Thesurrounding region of the pulmonary nodule may be a blurry edge, such asa thorn region, or other irregular region or curve. In some embodiments,the combination sub-unit 530 may combine a substance region and asurrounding region to determine a candidate target.

FIG. 6 is a flowchart illustrating an exemplary process for determininga candidate target by a candidate target determination unit 340according to some embodiments of the present disclosure.

In 610, pending data relating to an ROI may be acquired. Operation 610may be performed by the preprocessing unit 330.

In 620, preprocessed data may be determined by preprocessing the pendingdata of the ROI. Operation 620 may be performed by the preprocessingunit 330. In some embodiments, the preprocessing may include preliminarypositioning, enhancement processing, interpolation processing,morphology processing, denoising processing, or the like, or acombination thereof. The preliminary positioning may be performed todetermine a rough region of a candidate target in the ROI. Thepreliminary positioning may be performed automatically,semi-automatically, manually, etc. The enhancement processing may beperformed to highlight a structure or region in the ROI. The enhancementprocessing may include a Hessian dot-enhancement, a Hessianline-enhancement, or the like, or a combination thereof. Theinterpolation processing may be performed to balance voxel sizes in theROI. The morphology processing may be performed to analyze and identifya candidate target by processing a shape of a structure in the ROI basedon an element with a specific morphological structure. The morphologyprocessing may include an expansion operation, a corrosion operation, anopen operation, a closed operation, or the like, or a combinationthereof. The denoising operation may be performed to remove noise causedby machines and/or object motions in the ROI. The denoising operationmay employ a mean filtering, a Wiener filtering, a morphology filtering,a median filtering, or the like, or a combination thereof.

In 630, a substance region may be determined based on the preprocesseddata. Operation 630 may be performed by the substance regiondetermination sub-unit 510. In 640, a surrounding region may bedetermined based on the preprocessed data. Operation 640 may beperformed by the surrounding region determination sub-unit 510. In someembodiments, a segmentation model may correspond to a segmentationtechnique. In some embodiments, the substance region and/or thesurrounding region may be determined based on a segmentation model. Thesegmentation model may include a threshold segmentation model, a regiongrowing segmentation model, a watershed segmentation model, astatistical segmentation model, a morphological segmentation model, orthe like, or a combination thereof. The threshold segmentation modelaccording to threshold number may include a single thresholdsegmentation model, a multi-threshold segmentation model, etc. Thethreshold segmentation model according to an algorithm applied by themodel may include an iterative threshold segmentation model, a histogramsegmentation model, the Otsu segmentation model, etc. The morphologicalsegmentation model may include a Hessian dot-enhancement model, aHessian-line enhancement model, a multi-scale Gaussian template matchingmodel, a multi-scale morphological filtering model, etc. The edge-basedsegmentation model may include a level set algorithm. The statisticalsegmentation model may include a variational expectation maximization, aK-means model, a Fuzzy C-means model, or the like, or a combinationthereof.

In 650, a candidate target may be determined by combining the substanceregion and the surrounding region. Operation 650 may be performed by thecombination sub-unit 530. In some embodiments, the candidate target maybe a combination of the substance region and the surrounding region.

It should be noted that the flowchart for determining a candidate targetdepicted above is provided for the purposes of the illustration, and notintended to limit the scope of the present disclosure. For personshaving ordinary skills in the art, various variations and modificationmay be conducted under the teaching of the present disclosure. Forexample, operation 620 may be omitted, and operation 610 may beperformed by the substance region determination sub-unit 510 and/or thesurrounding region determination sub-unit 520. As another example,operations 630 and 640 may be performed without a specific order.Operation 630 may be performed first, operation 640 may be performedfirst, or operations 630 and 640 may be performed synchronously.However, those variations and modifications may not depart the scope ofthe present disclosure.

FIG. 7 is a flowchart illustrating an exemplary process for determininga candidate target by a candidate target determination unit 340according to some embodiments of the present disclosure. Thedetermination of the candidate target may be performed by the candidatetarget determination unit 340.

In 712, pending data of a region of interest (ROI) may be acquired.

In 714, a first positioning region may be determined by performingpreliminary positioning on the pending data of the ROI. In someembodiments, the first positioning region may include one or moreregions. In some embodiments, the preliminary positioning may beperformed to reduce the calculation amount and improve the efficiencyfor determining the candidate target. In some embodiments, thepreliminary positioning may be performed based on a substance region ofa candidate target. For example, in a process for detecting a pulmonarynodule, the first positioning region may be determined by performing thepreliminary positioning based on a shape of the substance region ofpulmonary nodule (e.g., circular or quasi-circular). The firstpositioning region may include a suspected pulmonary nodule, a bloodvessel, a pulmonary parenchyma, or the like, or a combination thereof.In some embodiments, the first positioning region may be represented bya first outer rectangle frame. For example, in a 3D lung CT image with asize of 256 mm*256 mm*200 mm, the first positioning region with a sizeof 35 mm*35 mm*35 mm may be determined by performing the preliminarypositioning. In some embodiments, the preliminary positioning may beperformed manually, semi-automatically, automatically, etc. For example,the preliminary positioning may be performed by drawing the firstpositioning region manually on the pending data. As another example, thepreliminary positioning may be performed by determining a long axis inthe pending data traversing the substance region of the candidatetarget, selecting one or more seed points based on the long axis, anddetermining the first positioning region based on a threshold-basedregion growing technique. As still another example, the firstpositioning region may be determined by performing the preliminarypositioning based on a specific algorithm automatically.

In 720, a second positioning region may be determined by performing athreshold segmentation on the first positioning region based on Hessiandot-enhancement. In some embodiments, the threshold segmentation basedon Hessian dot-enhancement may be performed to enhance a circular orquasi-circular (or spherical or quasi-spherical in a 3D image)structure, decrease a region for determining a candidate target, andimprove the accuracy and speed for determining the candidate target. Forexample, in a process for detecting a pulmonary nodule, because of thesimilarity of gray scale distribution corresponding to pulmonary bloodvessels, bronchus, and pulmonary nodules in a CT image, it is easy tomake a misdiagnosis or missed diagnosis to the pulmonary nodule. Infact, blood vessels, bronchus, and pulmonary nodules may be different inthe spatial morphology. For example, the pulmonary blood vessels and thebronchus may be in a tube structure, and the pulmonary nodules mayinclude a substance region and a surrounding region. The substanceregion may be circular or quasi-circular (or spherical or spherical in a3D image). Therefore, the second positioning region may be determined byperforming a threshold segmentation based on Hessian dot-enhancement fordistinguishing the substance region of pulmonary nodule from noises(e.g., the pulmonary blood vessels, the bronchus, etc.) by enhancing thesubstance region of pulmonary nodule.

For illustration purposes, a threshold segmentation based on a Hessiandot-enhancement may be performed on a 3D image as an example. Thethreshold segmentation based on Hessian dot-enhancement may include:performing a Gaussian smoothing on the 3D image, determining a secondderivative of each voxel value denoted by f (x, y, z) in the 3D image,the voxel value representing a gray value of the voxel, generating aHessian matrix based on second derivatives of the voxel value ondifferent directions, determining proper values of the Hessian matrix,and determining a dot enhanced value by adding the proper value toEquation (2). The dot enhanced value may represent a voxel valuedetermined by processing the voxel by the threshold segmentation basedon Hessian dot-enhancement. The Hessian matrix and the voxel valuedetermined by the threshold segmentation based on Hessiandot-enhancement may be determined according to Equations (1) and (2):

$\begin{matrix}{{H = \begin{bmatrix}f_{xx} & f_{xy} & f_{xz} \\f_{yx} & f_{yy} & f_{yz} \\f_{zx} & f_{zy} & f_{zz}\end{bmatrix}},} & (1) \\{and} & \; \\{{z_{dot}\left( {\lambda_{1},\lambda_{2},\lambda_{3}} \right)} = \left\{ {\begin{matrix}{{{\lambda_{3}}^{2}/\lambda_{1}},{{{if}\mspace{14mu}\lambda_{1}} < 0},{\lambda_{2} < 0},{\lambda_{3} < 0}} \\{0,{other}}\end{matrix},} \right.} & (2)\end{matrix}$wherein f_(xx) refers to a second derivative of a voxel on X direction,f_(xy) refers to a second derivative on Y direction of the firstderivative on X direction, λ₁, λ₂, and λ₃ refer to proper values of theHessian matrix, z_(dot) refers to a voxel value determined by processinga voxel by the threshold segmentation based on Hessian dot-enhancement.

In some embodiments, the second positioning region may include acandidate target and a background region. The background region mayinclude a structure that does not belong to the candidate target. Forexample, in a process for detecting a pulmonary nodule, the backgroundregion may include pulmonary blood vessels, bronchus, etc. In someembodiments, the second positioning region may be represented by asecond outer rectangle frame. In some embodiments, the area of the firstouter rectangle may be less than the area of the second outer rectangle.

It should be noted that the description for the threshold segmentationbased on a Hessian dot-enhancement depicted above is provided for thepurposes of the illustration, and not intended to limit the scope of thepresent disclosure. For persons having ordinary skills in the art,various variations and modification may be conducted under the teachingof the present disclosure. However, those variations and modificationsmay not depart the scope of the present disclosure. For example, thethreshold segmentation based on Hessian dot-enhancement may be performedon a 2D image.

In 722, a probability graph may be determined by processing the secondpositioning region based on a statistical segmentation model. In someembodiments, the statistical segmentation model may be a segmentationmodel that is not based on the morphology. Candidate targets indifferent shapes may be segmented based on the statistical segmentationmodel. The statistical segmentation model may classify pixels or voxelsin an image into at least two types. For example, in a process fordetecting a pulmonary nodule, in a 3D image, the statisticalsegmentation model may be performed to determine whether a voxel is apulmonary nodule. In some embodiments, the statistical segmentationmodel may include a variational expectation maximization model. In someembodiments, the probability graph may be determined by processing thesecond positioning region based on the variational expectationmaximization model. The determination of the probability graph byperforming the variational expectation maximization model on the secondpositioning region may include that voxels in the candidate target andthe background region may comply with a Gaussian distribution, the meanvalue and the standard deviation of the Gaussian distribution may complywith a Gaussian-Wishar distribution, and the prior probability of theGaussian distribution may comply with a Dirichlet distribution. TheGaussian-Wishart distribution and the Dirichlet distribution may beillustrated by Equations (3) and (4) below:p(μ,Λ)=p(μ|Λ)p(Λ)=Π_(k=1) ^(K) N(μ_(k) |m ₀,(β₀Λ_(k))⁻¹)W(Λ_(k) |w ₀ ,v₀),  (3)andp(Ω)=Dir(π|α₀)=C(α₀)Π_(k=1) ^(K)π_(k) ^(α) ⁰ ⁻¹.  (4)where π refers to a random variable that is a prior probability of theGaussian mixing distribution, α₀ is a constant, k refers to adistribution number of the Gaussian distribution, μ refers to a meanvalue, and Λ refers to a variance. A joint probability density functionincluding five random variables may be illustrated by Equation (5). p(X,Z, π, μ, Λ) may be determined by determining α_(k), β₀, m_(k), w_(k) andv_(k) based on the variational expectation maximization model. Then theprobability graph of the second positioning region may be determined bydetermining probabilities of pixels or voxels in the second positioningregion. The joint probability density function may be illustrated byEquation (5) below:p(X,Z,π,μ,Λ)=p(X|Z,μ,Λ)p(Z|π)p(π)p(μ|Λ)p(Λ).  (5)

In some embodiments, the spatial positions of pixels or voxels in theprobability graph may correspond to the spatial positions of pixels orvoxels in the second positioning region. The values of pixels or voxelsin the probability graph may represent probabilities of which pixels orvoxels to belong to a candidate target.

In 724, the candidate target determination unit 340 may determinewhether a probability value of a pixel or voxel in the probability graphis greater than a first threshold and whether a gray value of the pixelor voxel in the second positioning region is greater than a secondthreshold. If the probability value of the pixel or voxel in theprobability graph is greater than the first threshold and the gray valueof the pixel or voxel in the second positioning region is greater thanthe second threshold, the process may proceed to 726. In 726, the pixelor voxel may be determined that the pixel or voxel belongs to thesubstance region. If the probability value of the pixel or voxel in theprobability graph is less than or equal to the first threshold and thegray value of the pixel or voxel in the second positioning region isless than or equal to the second threshold, the process may proceed to734. In 734, the pixel or voxel may be determined that the pixel orvoxel belongs to the surrounding region. In some embodiments, the firstthreshold may be in a range from 0 to 1. For example, the firstthreshold may be 0.5. In some embodiments, the second threshold may beset for protecting the substance region of the candidate target. Thesecond threshold may be in a range from 0 to 100. In some embodiments, apixel or voxel value may represent a color value of the pixel or voxelin an image. For a monochrome image, a pixel or voxel value mayrepresent a gray value of the pixel or voxel in an image.

In 728, a Hessian line-enhancement graph may be determined by performingHessian line-enhancement on the second positioning region. The Hessianline-enhancement may be performed to enhance a linear morphologicalstructure in an image. For example, in a process for detecting apulmonary nodule, the Hessian line-enhancement may be performed toenhance a tube structure (e.g., blood vessels) in the second positioningregion by highlighting the noise (e.g., blood vessels).

For illustration purposes, the Hessian-line enhancement may be performedon a 3D image as an example. The Hessian-line enhancement may include:performing a Gaussian smoothing on the 3D image, determining a secondderivative of each voxel value f (x, y, z) in the 3D image, the voxelvalue representing the gray value of a voxel, generating a Hessianmatrix based on second derivatives of each voxel value on differentdirections, determining proper values of the Hessian matrix, anddetermining a line enhanced value by adding the proper values toEquation (6). The line enhanced value may represent a voxel valuedetermined by processing the voxel by the Hessian line-enhancement. Thevoxel value corresponding to a voxel processed by the Hessianline-enhancement may be determined according to Equations (1) and (6)below:

$\begin{matrix}{{z_{line}\left( {\lambda_{1},\lambda_{2},\lambda_{3}} \right)} = \left\{ {\begin{matrix}{{\lambda_{2}\frac{{\lambda_{2}} - {\lambda_{3}}}{\lambda_{1}}},{{{if}\mspace{14mu}\lambda_{1}} < 0},{\lambda_{2} < 0},{\lambda_{3} < 0}} \\{0,{other}}\end{matrix},} \right.} & (6)\end{matrix}$where λ₁, λ₂, and λ₃ refer to proper values of Hessian matrix, andz_(line) refers to a voxel value determined by processing a voxel by theHessian line-enhancement.

It should be noted that the description for the Hessian line-enhancementdepicted above is provided for the purposes of the illustration, and notintended to limit the scope of the present disclosure. For personshaving ordinary skills in the art, various variations and modificationmay be conducted under the teaching of the present disclosure. However,those variations and modifications may not depart the scope of thepresent disclosure. For example, the Hessian line-enhancement may beperformed on a 2D image.

In 730, the candidate target determination unit 340 may determinewhether a probability value of a pixel or voxel in the probability graphis greater than the first threshold and whether a gray value of thepixel or voxel in the Hessian line-enhancement graph is less than athird threshold. If the probability value of the pixel or voxel in theprobability graph is greater than the first threshold and the gray valueof the pixel or voxel in the Hessian line-enhancement graph is less thanthe third threshold, the process may proceed to 732. In 732, the pixelor voxel may be determined that the pixel or voxel belongs to thesurrounding region. If the probability value of the pixel or voxel inthe probability graph is less than or equal to the first threshold andthe gray value of the pixel or voxel in the Hessian line-enhancementgraph is equal to or greater than the third threshold, the process mayproceed to 734. In 734, the pixel or voxel may be determined that thepixel or voxel belongs to the background region. In some embodiments,the first threshold may be in a range from 0 to 1. For example, thefirst threshold may be 0.5. In some embodiments, the third threshold maybe set for denoising and protecting the surrounding region of thecandidate target in irregular shape. For example, in a process fordetecting a pulmonary nodule, the third threshold may be set forremoving noise (e.g., blood vessels, bronchus, etc.) and reservenon-spherical pulmonary nodules in irregular shape. The third thresholdmay be in a range from 0 to 100.

In 736, a candidate target may be determined by combining the substanceregion and the surrounding region.

In the process for determining the candidate target as described in FIG.7, the first positioning region and the second positioning region may bedetermined based on the preliminary positioning and the thresholdsegmentation based on Hessian dot-enhancement. The initial form of thesubstance region of the candidate target may be enhanced, thecomputation volume may be reduced and the speed for determining thecandidate target may be improved. The substance region may be determinedbased on the second positioning region determined by the probabilitygraph and the threshold segmentation based on Hessian dot-enhancement.The surrounding region may be determined based on the probability graphand the Hessian line-dot enhancement. The candidate target may bedetermined by combining the substance region and the surrounding region.

It should be noted that the description for determining the candidatetarget depicted above is provided for the purposes of the illustration,and not intended to limit the scope of the present disclosure. Forpersons having ordinary skills in the art, various variations andmodification may be conducted under the teaching of the presentdisclosure. For example, operation 714 may be omitted. However, thosevariations and modifications may not depart the scope of the presentdisclosure.

FIG. 8 is a block diagram illustrating an example of a candidate targetdetermination unit 340 according to some embodiments of the presentdisclosure. The candidate target determination unit 340 may include afirst target determination sub-unit 810 and a second targetdetermination sub-unit 820. The first target determination sub-unit 810may determine a candidate target based on a first segmentation model.The second target determination sub-unit 820 may determine a candidatetarget based on a second segmentation model. In some embodiments, thefirst segmentation model and/or the second segmentation model mayinclude one or more segmentation models. In some embodiments, asegmentation model may correspond to a segmentation technique. Thesegmentation model may include a threshold segmentation model, a regiongrowing segmentation model, a watershed segmentation model, astatistical segmentation model, a morphological segmentation model, orthe like, or a combination thereof. The threshold segmentation modelaccording to threshold number may include a single thresholdsegmentation model, a multi-threshold segmentation model, etc. Thethreshold segmentation model according to algorithm may include aniterative threshold segmentation model, a histogram segmentation model,the Otsu segmentation model, etc. The morphological segmentation modelmay include a Hessian dot-enhancement model, a Hessian-line enhancementmodel, a multi-scale Gaussian template matching model, a multi-scalemorphological filtering model, etc. The edge-based segmentation modelmay include a level set algorithm. The statistical segmentation modelmay include a variational expectation maximization (VEM) model, aK-means model, a Fuzzy C-means model, or the like, or a combinationthereof. In some embodiments, the first target determination sub-unit810 may apply a morphological segmentation model (e.g., a fixed regiongrowing algorithm). The morphological segmentation model may beperformed based on gray values of pixels or voxels. The segmentationtechnique based on a morphological segmentation model may segment atarget based on gray values of pixels or voxels. The second targetdetermination sub-unit 820 may apply a statistical segmentation model(e.g., a K-means model, a Fuzzy C-means model, a VEM model, etc.). Thestatistical segmentation model may be performed based on a distributionof gray values.

It should be noted that the description for the candidate targetdetermination unit 340 depicted above is provided for the purposes ofthe illustration, and not intended to limit the scope of the presentdisclosure. For persons having ordinary skills in the art, variousvariations and modification may be conducted under the teaching of thepresent disclosure. For example, the candidate target determination unit340 may include more than two target determination sub-units fordetermining a candidate target based on different segmentation models.However, those variations and modifications may not depart the scope ofthe present disclosure.

FIG. 9 is a flowchart illustrating an exemplary process for determininga reference subset of features according to some embodiments of thepresent disclosure. The reference subset of features may be determinedby selecting features from a universal set of features. The universalset of features may include feature data of a candidate target asextensive as possible. The feature data important for training aclassifier may be selected to form the reference subset of features, sothat the classifier may detect and diagnose quickly and accurately.

In 910, pending data may be acquired. Operation 910 may be performed bythe preprocessing unit 330. The pending data may include original data,or a ROI determined based on the original data.

In 920, preprocessed data may be determined by preprocessing the pendingdata. Operation 920 may be performed by the preprocessing unit 330. Insome embodiments, the preprocessing may include preliminary positioning,enhancement processing, interpolation processing, morphology processing,denoising processing, or the like, or a combination thereof. Thepreliminary positioning may be performed to determine a rough region ofa candidate target in the ROI. The preliminary positioning may beperformed automatically, semi-automatically, manually, etc. Theenhancement processing may be performed to highlight a structure orregion in the ROI. The enhancement processing may include a Hessiandot-enhancement, a Hessian line-enhancement, or the like, or acombination thereof. The Hessian dot-enhancement may be performed toenhance, for example, a circular or quasi-circular (or spherical orquasi-spherical in a 3D image) structure. The Hessian line-enhancementmay be performed to enhance, for example, a linear structure. Forexample, in a process for detecting a pulmonary nodule, the pulmonarynodule may be circular or quasi-circular (or spherical orquasi-spherical in a 3D image). The Hessian dot-enhancement may beperformed to enhance a circular or quasi-circular (or spherical orquasi-spherical in a 3D image) candidate pulmonary nodule. The Hessianline-enhancement may be performed to enhance a linear noise (e.g., bloodvessels, bronchus, etc.). The candidate pulmonary nodule may bedistinguished from other structures. The interpolation processing may beperformed to generate isotropy or approximatively isotropy voxels. Forexample, the Hessian dot-enhancement or the Hessian line-enhancement maybe performed on an image with an equal distance between two adjacentpixels or voxels, such as an interpolated 3D image with voxels in sizeof 1 mm*1 mm*1 mm. The morphology processing may be performed to analyzeand identify a candidate target by processing a shape of a structure inthe ROI based on an element with a specific morphological structure. Themorphology processing may include filling an opening or gap in a pendingstructure. For example, in a process for detecting a pulmonary nodule, acavitation or bubble in the pulmonary nodule may not be identified basedon a segmentation process. The morphology processing may include fillingan opening or gap in the pulmonary nodule. The morphology processing mayinclude an expansion operation, a corrosion operation, an openoperation, a closed operation, or the like, or a combination thereof.The denoising operation may be performed to remove noise caused bymachines and/or object motions in the ROI. The denoising operation mayemploy a mean filtering, a Wiener filtering, a morphology filtering, amedian filtering, or the like, or a combination thereof.

In 930, a first candidate target may be determined by processing thepending data or the preprocessed data based on a first segmentationmodel. In 940, a second candidate target may be determined by processingthe pending data or the preprocessed data based on a second segmentationmodel. Operations 930 and 940 may be performed by the first targetdetermination sub-unit 810 and the second target determination sub-unit820, respectively. In some embodiments, the first segmentation modeland/or the second segmentation model may include one or moresegmentation models. In some embodiments, a segmentation model maycorrespond to a segmentation technique. The segmentation model mayinclude a threshold segmentation model, a region growing segmentationmodel, a watershed segmentation model, a statistical segmentation model,a morphological segmentation model, or the like, or a combinationthereof. The threshold segmentation model according to threshold numbermay include a single threshold segmentation model, a multi-thresholdsegmentation model, etc. The threshold segmentation model according toalgorithm may include an iterative threshold segmentation model, ahistogram segmentation model, the Otsu segmentation model, etc. Themorphological segmentation model may include a Hessian dot-enhancementmodel, a Hessian-line enhancement model, a multi-scale Gaussian templatematching model, a multi-scale morphological filtering model, etc. Theedge-based segmentation model may include a level set algorithm. Thestatistical segmentation model may include a variational expectationmaximization (VEM) model, a K-means model, a Fuzzy C-means model, or thelike, or a combination thereof.

In some embodiments, the first segmentation model may include amorphological segmentation model, such as a fixed threshold regiongrowing model. The morphological segmentation model may be performedbased on gray values of pixels or voxels. The second segmentation modelmay include a statistical segmentation model (e.g., a K-means model, aFuzzy C-means model, a VEM model, etc.). The statistical segmentationmodel may be performed based on a distribution of gray values. In someembodiments, the first segmentation model may be a fixed region growingmodel based on Hessian enhancement. In some embodiments, the fixedregion growing model based on Hessian enhancement may be a roughsegmentation model. The statistical segmentation model (e.g., a VEMmodel) may be an accurate model.

In some embodiments, one or more candidate targets may be determined byperforming the first segmentation model and the second segmentationmodel on a same region for multiple morphological structures in animage. More feature data may be extracted to improve the determinationof a reference subset of features and the training of a classifier. Forexample, in a process for detecting a pulmonary nodule, for a 3D image,a fixed threshold region growing model based on a Hessiandot-enhancement may be performed on a spherical or quasi-sphericalpulmonary nodule for determining a suspected pulmonary nodule. A fixedthreshold region growing model based on a Hessian line-enhancement maybe performed on a false positive source, for example, tubular bloodvessels or bone tissues, for determining a suspected pulmonary nodule.In some embodiments, the emphasis of different segmentation models maybe different. For example, a morphological segmentation model (e.g., aregion growing segmentation model) may segment an image based on grayvalues of pixels or voxels and determine a segmented result with aconstant morphology. A statistical segmentation model (e.g., a VEMmodel) may segment an image based on a distribution of gray values.Therefore, candidate targets determined based on different segmentationmodels may be different for a same region. The different candidatetargets may supplement mutually. The universal set of features composedby a union set of feature data corresponding to different candidatetargets may contain more effective feature information and expand therange of the universal set of features to improve the determination of areference subset of features and the training of a classifier.

In some embodiments, one or more candidate targets may be determined byperforming more than two segmentation models on a same region. Thefeature data related to the candidate targets may be selected from thecandidate targets. A universal set of features may include feature dataextracted from the candidate target as extensive as possible.

In 950, a universal set of features may be determined by extractingfeatures from the first candidate target and the second candidatetarget. Operation 950 may be performed by the first feature extractionunit 351. In some embodiments, the features may be extracted forclassifying a target from other targets based on measuring and/orprocessing the target. For example, in a process for detecting apulmonary nodule, the candidate target may be a suspected pulmonarynodule. The extracted features may include a gray value related feature,a morphological feature, a texture feature, a serial slice feature, orthe like, or a combination thereof. The gray value related feature mayinclude a mean gray value, a gray value variance, or the like, or acombination thereof. In a 2D image, the morphological feature mayinclude may include an area, a perimeter, a centroid, a diameter, acurvature, an eccentricity, a roundness, a compactness, a Fourierdescriptor, a shape moment descriptor, or the like, or a combinationthereof. In a 3D image, the morphological feature may include a volume,a perimeter, a centroid, a diameter, a curvature, an eccentricity, asphericity, a compactness, a Fourier descriptor, a shape momentdescriptor, or the like, or a combination thereof. The texture featuremay include a statistic magnitude, a boundary clarity factor, afrequency domain parameter, etc.

In 950, the features may be extracted based on the pending data or thepreprocessed data. In some embodiments, the preprocessed data mayinclude an interpolation image, a Hessian dot-enhanced image, a Hessianline-enhancement image, a morphology image, or the like, or acombination thereof. In some embodiments, the universal set of featuresmay include feature data extracted from the first candidate target andthe second candidate target, the pending data, or the preprocessed data.In addition, the universal set of features may include non-image data.The non-image data may include medical history data, clinical trialresults, symptoms, physiological data, pathological data, genetic data,or the like, or a combination thereof. For example, the non-image datafor a pulmonary nodule detection may include age, gender, smokinghistory, cancer history, family history, occupation exposure, leisureexposure, previous lung disease, current lung disease, previous thoracicsurgery, the number of satellite lesions around diagnosed lesions, lymphnode size, suspicious lesions, lesions location in the lung, or thelike, or a combination thereof.

In 960, a reference subset of features may be determined based onselecting features from the universal set of features. Operation 960 maybe performed by the feature selection unit 360. In some embodiments, theselection of features may improve the accuracy of a classifier. In someembodiments, the selection of features may include selecting importantfeatures from the universal set of features for distinguishing differenttypes according to different classify purposes, and rejectingnon-effective features for the classification. For example, in a processfor detecting a pulmonary nodule, the classification may be performed todetermine whether a suspected pulmonary nodule is a true pulmonarynodule. The important features for determining a true pulmonary nodule(e.g., a mean gray value, a gray variance, etc.) may be selected and thefeatures having no effect on determining a true pulmonary nodule (e.g.,a frequency domain parameter feature) may be rejected. In a process fordiagnosing a pulmonary nodule, the classification may be performed todetermine whether the pulmonary nodule is benign or malignant. Theimportant features for determining whether the pulmonary nodule isbenign or malignant (e.g., a boundary clear degree factor) may beselected and the features having no effect on determining whether thepulmonary nodule is benign or malignant (e.g., a degree of circularityor sphericity) may be rejected. The features may be selected based on anexhaustion algorithm, a heuristic algorithm, a random algorithm, or thelike, or a combination thereof. The random algorithm may include agenetic algorithm, a particle swarm algorithm, a simulated annealingalgorithm, etc. The heuristic algorithm may include a breadth firstsearch algorithm, a depth first search algorithm, etc. The heuristicalgorithm may include a decision tree algorithm, a Relief algorithm, aforward (backward) selection algorithm, etc.

In some embodiments, Operations 910-940 may be performed to determine acandidate target. Operations 910-960 may be performed to determine areference subset of features in a process for training a classifier.

It should be noted that the description for selecting features depictedabove is provided for the purposes of the illustration, and not intendedto limit the scope of the present disclosure. For persons havingordinary skills in the art, various variations and modification may beconducted under the teaching of the present disclosure. However, thosevariations and modifications may not depart the scope of the presentdisclosure. For example, Operation 920 may be omitted. Then Operation910 may be performed by the first target determination sub-unit 810and/or the second target determination sub-unit 820.

FIG. 10 is a schematic diagram for illustrating exemplary featuresources of feature extraction according to some embodiments of thepresent disclosure. Different preprocessed data may be determined basedon performing different preprocessing on an original image. Thepreprocessed data may include an interpolation image, a Hessiandot-enhanced image, a Hessian line-enhancement image, a morphologyimage, or the like, or a combination thereof. Different candidatetargets may be determined by performing different segmentation models onthe preprocessed data. Features may be extracted from the originalimage, the preprocessed data, the candidate targets, or the like, or acombination thereof.

In some embodiments, S1 may represent an original medical image. Theoriginal medical image may include an X-rays image, a CT image, a PETimage, an MRI image, an ultrasonic image, an electrocardiogram, anelectroencephalogram, or the like, or a combination thereof. Theoriginal medical image may be 2D or 3D.

S2 may represent an interpolation image. The interpolation image may bedetermined by performing interpolation processing on the originalmedical image S1. The interpolation image S2 may provide an image withan equal distance in different directions for follow-up processing, forexample, a Hessian dot-enhancement processing and a Hessianline-enhancement processing. For example, voxels in a 3D originalmedical image may be in a size of 1 mm*1 mm*1 mm based on interpolationprocessing.

S3 may represent a morphology image. The morphology image may bedetermined by performing morphological processing on the originalmedical image S1. The morphological processing may include an expansionoperation, a corrosion operation, an open operation, a closed operation,or the like, or a combination thereof. The morphological processing maybe performed to fill an opening or hole in a candidate target. Forexample, a cavitation or bubble (also referred to as an opening or hole)in a pulmonary nodule may not be identified based on a segmentationtechnique. The morphological processing may be performed to fillopenings or holes in the pulmonary nodule.

A Hessian dot-enhanced image S4 and a Hessian line-enhanced image S5 maybe determined by performing a Hessian dot-enhancement and a Hessianline-enhancement on the interpolation image S2. A segmented image S6 anda segmented image S7 may be determined by performing a fixed thresholdregion growing model on the Hessian dot-enhanced image S4 and theHessian line-enhanced image S5, respectively. A VEM segmented image S8may be determined by performing a VEM model on the medical image S1. Animage S9 and an image S10 may be determined by performing a Hessiandot-enhancement and a Hessian line-enhancement on the VEM segmentedimage S8, respectively. A segmented image S11 may be determined byperforming a fixed threshold region growing model on the segmented imageS11. A segmented image S12 and a segmented image S13 may be determinedby performing a fixed threshold region growing model and a VEM modelrespectively on the interpolation image S2. The images described abovemay supplement feature information related to a candidate target toimprove a relevance between the feature data in a universal set offeatures and the candidate target, and improve the accuracy for traininga classifier by using the feature data in the universal set of features.

It should be noted that the description for feature sources depictedabove is provided for the purposes of the illustration, and not intendedto limit the scope of the present disclosure. For persons havingordinary skills in the art, various variations and modification may beconducted under the teaching of the present disclosure. However, thosevariations and modifications may not depart the scope of the presentdisclosure. The feature sources may not be limited to the imagesdescribed above. Feature sources may be determined by transforming acombination of the segmentation models and the preprocessing operations,and types and numbers of segmentation models or preprocessingoperations. For example, a candidate target may be determined byperforming a morphological processing, a Hessian enhancement, amorphological segmentation model (e.g., a fixed threshold region growingmodel), and a statistical segmentation model (e.g., a VEM model) on theinterpolation image S2.

FIG. 11 is a flowchart illustrating an exemplary process for determininga reference subset of features based on a simulated annealing algorithmaccording to some embodiments of the present disclosure. The process fordetermining a reference subset of features may be performed by thefeature selection unit 360. In some embodiments, the simulated annealingalgorithm may be a probability algorithm for searching for an optimalsolution in a large searching space. Using a simulated annealingalgorithm, an initial value and a large enough initial temperature maybe set. An optimal solution may be searched by performing iterationoperations based on the initial value. At the initial temperature, if aspecific number of iteration operations is finished, the initialtemperature may be decreased to a new temperature slowly. Then a nextnumber of iteration operations may be performed at the new temperature.If the temperature is decreased to a minimum value, the iterationoperations may terminate, and the current optimal solution may be afinal solution.

In 1110, a temperature T may be set. If the temperature T is an initialtemperature, the initial temperature may be large enough to satisfy acondition for a comprehensive searching in the simulated annealingalgorithm.

In 1120, a subset of features i and a score y(i) of the subset offeatures i may be selected from a universal set of features. In someembodiments, the subset of features i may include any combination offeature data in the universal set of features. The subset of features imay include one or more feature data. For example, the subset offeatures i may include an area, a gray mean, or the like, or acombination thereof. In some embodiments, the score y(i) of the subsetof features may be determined based on the free-response receiveroperating feature curve (FROC) or an area under a receiver operatingfeature curve (ROC). In some embodiments, the score y(i) of the subsetof features may be determined by an area under a classifier curvetrained by feature data in the subset of features i.

In 1130, a neighborhood subset of features j of the subset of features iand a score y(j) of the neighborhood subset of features j may beselected from the universal set of features. In some embodiments, thesubset of features j may include an area, a gray mean, a gray variance,a compactness, or the like, or a combination thereof.

In 1140, the feature selection unit 360 may determine whether a scoredifference value δ between the score y(i) and the score y(j) (alsoreferred to as δ=y(i)−y(j)) is less than 0. If δ<0, the process mayproceed to 1190, and i may be set to be equal to j. If δ≥0, the processmay proceed to 1150. Operation 1140 may be performed to compare thescore of the subset of features i and the score of the subset offeatures j. As used herein, the score of the subset of features i andthe score of the subset of features j may be also referred to asaccuracy rate scores of classifiers determined by applying the featuredata in the subset of features i and the subset of features j to trainthe classifiers. If the score of the subset of features j is greaterthan the score of the subset of features i, the accuracy rate of theclassifier trained by the feature data in the subset of features j isgreater than the accuracy rate of the classifier trained by the featuredata in the subset of features i. Thereof, the subset of features j maybe determined as a current solution to substitute the subset of featuresi for a next iteration.

In 1150, a probability P may be determined based on a formula exp(−δ/T),and whether the probability P is greater than a random value in a rangefrom 0 to 1 may be determined. If the probability P is greater than therandom value in the range from 0 to 1, the process may proceed to 1190,and i may be set to be j. If the probability P is less than or equal tothe random value in the range from 0 to 1, the process may proceed to1160. In some embodiments, the random value may be determined randomlyor based on an iteration condition (e.g., an iteration temperature). Therandom value may be set to avoid a local optimal solution by receivingthe subset of features j as a next iteration solution when the score ofthe subset of features j is less than the score of the subset offeatures i. If the score of the subset of features j is less than thescore of the subset of features i, the subset of features j may bedetermined as a current solution with a probability P. If theprobability P is greater than a random value in a range from 0 to 1, thesubset of features j may be determined as a current solution. If theprobability P is less than or equal to the random value in the rangefrom 0 to 1, the subset of features j may not be determined as a currentsolution.

In 1160, whether the search is sufficient in the temperature T may bedetermined. If the search is non-sufficient, the process may proceed to1120. If the search is sufficient, the process may proceed to 1170. Insome embodiments, the sufficient search may mean that the amount ofiterations reaches a preset value in the temperature T. In someembodiments, the sufficient search may also mean that an optimalsolution searched in the temperature T may keep unchanged under severalcontinuous iteration operations.

In 1170, whether the process meets a termination condition may bedetermined. If the process meets the termination condition, the processmay proceed to 1180. In 1180, a subset of features with a maximum scoremay be determined as a reference subset of features. If the process doesnot meet the termination condition, the process may proceed to 1110. Thetemperature T may be reset and operations 1110-1180 may be iterated. Insome embodiments, the termination condition may include that the currentiteration temperature may be less than or equal to the preset minimumtemperature, the iteration temperature may be stable, the amount ofiterations may reach a preset value, the optimal solution may be keepunchanged under several continuous iteration operations, etc. In someembodiments, the preset minimum temperature may be 0 degree. In someembodiments, the reset temperature may be lower than the lasttemperature for meeting a condition of the simulated annealingalgorithm. In some embodiments, the set of temperature for eachiteration may affect the results of the simulated annealing algorithm.For example, if the annealing temperature declines slowly, the searchingspeed may drop greatly, and the possibility of searching for a globallyoptimal solution may be larger. If the annealing temperature declinesquickly, the searching speed may be fast, and a globally optimalsolution may be searched inadequately. In some embodiments, thetemperature may be set based on experience or an annealing function. Theannealing function may be used to set a temperature and control theannealing speed. The annealing function may include Equations (7), (8),(9), etc.:

$\begin{matrix}{{{T_{1}(t)} = \frac{T_{01}}{\ln\left( {1 + t_{1}} \right)}},} & (7) \\{or} & \; \\{{{T_{2}(t)} = \frac{T_{02}}{\ln\left( {1 + {a_{2}t_{2}}} \right)}},} & (8) \\{or} & \; \\{{{T_{3}(t)} = {T_{03}a_{3}^{t_{3}}}},} & (9)\end{matrix}$wherein T₁(t), T₂(t), and T₃(t) represent different annealing functions,T₀₁, T₀₂, and T₀₃ represent initial temperatures, t₁, t₂, and t₃represent the number of times for which the temperature declines, a₂ anda₃ are adjustable parameters that may be used to adjust the annealingspeed.

In some embodiments, if the score different value δ is equal toy(j)−y(i), the simulated annealing algorithm may be performed todetermine a subset of feature with a minimum score as a reference subsetof features. In some embodiments, the simulated annealing algorithm maybe performed to avoid a local optimal solution. At each temperature, thesimulated annealing algorithm may be performed with a number ofneighborhood motions. Then the temperature may be dropped slowly to makethe accuracy rate of a classifier trained by the feature data in thereference subset of features as higher as possible.

FIG. 12 is a flowchart illustrating an exemplary process for segmentinga region in real time according to some embodiments of the presentdisclosure. The process may be performed to segmenting a target regionin a medical image in real time. The target region may be positioned andone or more segmentation parameters may be adjusted synchronously. Auser may control a segmentation result according to different conditionsin real time. The process may be performed to determine a ROI in thedisclosure. For example, the process may be performed to determine alung image by separating the left and right lung regions from weasandsand bronchi in a chest CT image. As another example, the process may beperformed to determine a breast image from a chest X-rays image. Theprocess may be performed to determine a candidate target in thedisclosure. For example, the process may be performed to determine asuspected pulmonary nodule in a process for detecting a pulmonarynodule, or determine a calcification or a breast lump region in aprocess for detecting a breast. The process may be performed todetermine a first positioning region by preliminary positioning in thedisclosure. The process may be performed to determine a secondpositioning region in the disclosure.

In 1210, a pending region may be determined. In some embodiments, thepending region may be drawn in a medical image manually by a user basedon experience. In some embodiments, the pending region may be determinedbased on a hardware (e.g., a computer processor), a software (e.g., acomputer graphic processing), or a combination thereof.

In 1220, a starting point and a terminal point in the pending region maybe determined. In some embodiments, the starting point and the terminalpoint may be determined manually, semi-automatically, automatically, orthe like, or a combination thereof. For example, the starting point andthe terminal point may be determined by moving a mouse manually or bytouching a screen manually in the pending region.

In 1240, a target region and a segmentation parameter may be determinedbased on the starting point and the terminal point. The segmentationparameter may be a vertical component of a vector quantity from thestarting point and the terminal point. The segmentation parameter mayvary with a position of the terminal point. The segmentation parametermay be different under different image scenes. For example, thesegmentation parameter applied in a cucoloris image and an enhancedimage may be different for segmenting a breast MRI image in real time.The segmentation parameter may be adjusted according to clinic demands.The target region may be a circular region with a radius that may beequal to a horizontal component of the vector quantity from the startingpoint and the terminal point.

In 1250, whether the terminal point is out of the pending region may bedetermined. If the terminal point is within the pending region, theprocess may proceed to 1270. If the terminal point is out of the pendingregion, the process may proceed to 1260. In some embodiments, if thehorizontal projection of the vector quantity from the starting point andthe terminal point is within the pending region, the terminal point iswithin the pending region. If the horizontal projection of the vectorquantity from the starting point and the terminal point is out of thepending region, the terminal point is out of the pending region.

In 1260, the target region may be segmented in real time based on thesegmentation parameter and a technique of adaptive region growing. Insome embodiments, a lower threshold of the adaptive region growingtechnique may be determined according to Equation (10) below:th _(L)=½[(1−t)×(T _(max) +T _(min))],  (10)where th_(L) refers to the lower threshold of the adaptive regiongrowing technique, t refers to a segmentation parameter, T_(max) refersto a maximum gray value, and T_(min) refers to a minimum gray value. Asillustrated in Equation (10), the maximum gray value and the minimumgray value may be updated with the varying of the segmentationparameter, and may be in direct proportion to the segmentationparameter. The lower threshold may be adjusted based on the segmentationparameter. The smaller the segmentation parameter is, the smaller thelower threshold may be, the more the pixels or voxels contained in thetarget region may be, and the larger the growing region may be.

In 1270, the target region may be segmented in real time based on thesegmentation parameter and a technique of fixed threshold regiongrowing. In some embodiments, the lower threshold may be a minimum grayvalue of a pixel or voxel on a line connecting the start point and theterminal point.

FIG. 13 is a schematic diagram illustrating an exemplary segmentedresult based on a real-time segmentation technique according to someembodiments of the present disclosure. As shown, 1 represents a pendingregion, 2 represents a start point, 4 represents a terminal point, 3represents a target region determined by the start point 2 and theterminal point 4, and 5 represents a segmentation result of the pendingregion 1.

FIG. 14-A and FIG. 14-B are schematic diagrams illustrating exemplarysegmented results based on a real-time segmentation technique accordingto some embodiments of the present disclosure. As shown, a segmentationparameter is a vertical component 8 of a vector quantity 6 from astarting point 2 and a terminal point 4. A target region 3 is a circularregion with a radius that may be a horizontal component 7 of a vectorquantity from the starting point 2 and the terminal point 4. The targetregion 3 is segmented in real time by controlling the terminal point 4.If the terminal point 4 is within the pending region 1, the segmentedresult 5 may be determined by segmenting the target region 3 in realtime based on a technique of fixed threshold region growing shown inFIG. 14-A. If the terminal point 4 is out of the pending region 1, thesegmented result 5 may be determined by segmenting the target region 3in real time based on a technique of adaptive region growing shown inFIG. 14-B. Whether the terminal point 4 is within the pending region 1may be determined based on the horizontal projection of the vectorquantity 6 from the starting point 2 and the terminal point 4. If thehorizontal projection of the vector quantity 6 from the starting point 2and the terminal point 4 is within the pending region 1, the terminalpoint 4 is within the pending region 1. If the horizontal projection ofthe vector quantity 6 from the starting point 2 and the terminal point 4is out of the pending region 1, the terminal point 4 is out of thepending region 1.

FIG. 15 is a block diagram illustrating an example of a real-timestreaming parallel computing mode according to some embodiments of thepresent disclosure. In some embodiments, a server 130 may perform allprocesses in a serial computing mode by one single server or in areal-time streaming parallel computing mode (as shown in FIG. 15). Insome embodiments, a plurality of instructions may be performedsimultaneously in the parallel computing mode. A computing process maybe resolved into one or more portions, and the one or more portions maybe performed in a parallel computing mode. In some embodiments, in areal-time streaming mode, a procedure may be performed to monitor datageneration. For example, a portion of generated data may be transmittedto a real-time streaming computing system for processing, and theprocessed data may be outputted or stored in a database. In someembodiments, a real-time computing mode may, for example, collect,process, and analyze data by a lower delay time when the data vary inreal time. For example, in a CAD system applying a real-time streamingparallel computing mode, data from one or more examined objects (e.g.,multiple patients) or from different portions of one examined object maybe processed simultaneously. At least one portion of the detecting ordiagnosing result may be outputted. For example, when a portion of datais generated, the system may generate a detecting or diagnosing resultby processing the portion of data in a real-time streaming parallelcomputing mode. Then the detecting or diagnosing result may beoutputted.

As shown in FIG. 15, the server 130 for CAD in a real-time streamingparallel computing mode may include a message server cluster 1510, oneor more work nodes 1520 and a cluster management node 1530. The messageserver cluster 1510 may distribute pending data in a streaming mode. Thepending data may include an original medical image or intermediate data.The intermediate data may include an ROI, preprocessed data, a candidatetarget, feature data, or the like, or a combination thereof. The worknode 1520 may acquire the pending data distributed by the message servercluster 1510 and acquire a classification result by processing thepending data in a real-time streaming parallel computing mode. The worknode 1520 may acquire the pending data from the message server cluster1510 initiatively. The cluster management node 1530 may coordinate themessage server cluster 1510 and the work node 1520. The server 130 mayprocess lesions and store the processing result in a database or a filedevice.

In some embodiments, the message server cluster 1510 may include aScribe cluster, a Flume cluster, a Time Tunnel cluster, a Chukwacluster, Kafka cluster, or the like, or a combination thereof. Theserver 130 may include a Storm system, a Spark system, a Samza system, aS4 system, or the like, or a combination thereof. The cluster managementnode 1530 may include a zookeeper cluster.

For illustration purposes, a real-time streaming parallel computing modeperformed by the server 130 including a Zookeeper cluster, a Stormsystem, and a Kafka cluster may be illustrated as an example.

In some embodiments, the system may include a Storm frame cluster toprocess the pending data in a real-time streaming parallel computingmode. The Kafka cluster may release and subscribe messages indistributed mode with a high throughput. The Kafka cluster may classifyand process data, and transmit the processed data to the Storm system.In some embodiments, external data may be transmitted to the Stormsystem for processing by the Kafka cluster. The external data mayinclude an original medical image from a PACS system, a database, a filedevice, a medical imaging device, etc., or intermediate data (e.g., ROIdata, preprocessed data, candidate target data, etc.).

In some embodiments, the Storm system may be an open source, distributedand higher fault-tolerant real-time streaming computing system. In theStorm system, the chart structure for real-time computing may be alsoreferred to as a Topology. The Storm system may include one or moreTopologies (e.g., a Topology A 1521-1, a Topology B 1521-2, etc.). TheTopology may be submitted to clusters in the Storm system. A Mask nodein the clusters may distribute codes and tasks to a Work node. The Worknode may include one or more Supervisors. The Supervisors may monitorthe servers. One Supervisor may include one or more Workers. TheTopology may operate with one or more Workers. One Worker may includeone or more Executors for executing one or more Tasks. A Topology mayinclude one or more Spouts (e.g., a Spout A 1522-1, a Spout B 1522-2,etc.) and one or more data processing bolts (e.g., a node A 1523-1, anode B 1524-1, a node C 1525-1, a node D 1526-1, a node E 1523-2, a nodeF 1524-2, a node G 1525-2, a node H 1526-2, etc.). The Spouts mayacquire data from the Kafka cluster initiatively and send the streamingdata to the data processing nodes in Tuple form. The data processingnodes may transform the streaming data. For example, the data processingnodes may perform calculation operations and/or screening operations onthe streaming data. The data processing nodes may send the data to otherdata processing nodes. For example, the Spout A 1522-1 may send the datafrom the Kafka cluster to the node A 1523-1. The node A 1523-1 mayprocess the data and send the processed data to the node C 1525-1 forfurther processing. A Spout and/or a data processing node of a Topologymay correspond to multiple Workers distributed in multiple Supervisors.The data processed by the Storm system may be outputted to a databasefor storing.

In some embodiments, the Zookeeper cluster may be a distributedapplication coordinating service with opening source codes. TheZookeeper cluster may coordinate the Kafka cluster and the Storm system,and store public data (e.g., heartbeat information, cluster stateinformation or configuration information).

A detection process performed by the CAD system in real-time streamingparallel computing mode may be illustrated as an example. The detectionprocess may be performed by the CAD system in a real-time streamingparallel computing mode. In some embodiments, the detection process inthe CAD system may include a process for determining an ROI, a processfor preprocessing, a process for determining a candidate target, aprocess for extracting features, and a process for classifying. In someembodiments, the process for determining an ROI, the process forpreprocessing, and the process for determining a candidate target may beperformed by, for example, one single server. When the candidate targetis determined, the DICOM data may be sequenced into messages forentering to the Kafka cluster queue. The Spouts in one or moreTopologies of the Storm system may acquire the data initiatively fromthe Kafka cluster queue. The data processing bolts in the Topologies mayprocess the data in the real-time streaming mode to extract features andclassify the candidate target.

In some embodiments, the detection operations processed by the Stormsystem may be assigned in any manner. For example, the process fordetermining an ROI, the process for preprocessing, the process fordetermining a candidate target, and the process for extracting featuresmay be performed by, for example, one single server and the process forclassifying may be performed by the Storm system in a real-timestreaming parallel computing mode. For example, a portion of data may beprocessed by, for example, one single server for performing the processfor determining an ROI, the process for preprocessing, and the processto determine a candidate target, and by the Storm system in a real-timestreaming parallel computing mode for performing the process forextracting features and the process for classifying. Other portions ofdata may be processed by, for example, one single server for performingthe process for determining an ROI, the process for preprocessing, theprocess to determine a candidate target and the process for extractingfeatures, and by the Storm system in a real-time streaming parallelcomputing mode for performing the process for classifying. In someembodiments, original data may be sent to the Kafka cluster. The Stormsystem may perform the process for determining an ROI, the process forpreprocessing, the process for determining a candidate target, theprocess for extracting features, and the process for classifying in areal-time streaming parallel computing mode.

EXAMPLES

It should be noted that the description for embodiments depicted belowis provided for the purposes of the illustration, and not intended tolimit the scope of the present disclosure.

Example 1

FIG. 16-A is an exemplary original chest computed tomography (CT) imageobtained through scanning a human body by a computed tomography (CT)device according to some embodiments of the present disclosure. Theoriginal chest CT image includes a background region, lung tissues, fattissues, muscles, blood vessels, tracheas, skeletons, etc. Pulmonaryparenchyma may be extracted by processing the original chest CT image.FIG. 16-B is an exemplary mask image relating to a pulmonary parenchymaaccording to some present disclosure. The pulmonary parenchymarepresented in the image is bright. FIG. 16-C is an exemplary segmentedresult relating to a pulmonary nodule according to some embodiments ofthe present disclosure. The determined pulmonary nodule represented inthe image is bright.

Example 2

FIG. 17 is a schematic diagram illustrating an example of a diagnosisresult generated by a computer aided diagnosis (CAD) system according tosome present disclosure. As shown, the left part of the diagram is adetection result of a suspected pulmonary nodule and the right part is adiagnosis result as to whether the pulmonary nodule is benign ormalignant. The diagnosing result is 0.733 (also referred to as aprobability of being a malignant pulmonary nodule). The pulmonary nodulewas diagnosed to be malignant by the CAD system.

Example 3

FIGS. 18-A and 18-B are exemplary images of a breast lump regionsegmented by a region real-time segmentation algorithm according to somepresent disclosure. As shown in FIGS. 18-A and 18-B, a start point 2 wasdetermined as a seed point for positioning a target region. A terminalpoint 4 was selected by moving a mouse or touching a screen. A segmentedparameter and size of the target region were adjusted by controlling theterminal point 4. The segmented result of a breast lump was determinedin real time and represented in FIGS. 18-A and 18-B as the highlightedportions. The segmented result 9 as shown in FIG. 18-A was determined bya technique of a fixed threshold region growing corresponding to theterminal point within the pending region. The segmented result 10 asshown in FIG. 18-B was determined according to a technique of adaptiveregion growing corresponding to the terminal point out of the pendingregion.

Having thus described the basic concepts, it may be rather apparent tothose skilled in the art after reading this detailed disclosure that theforegoing detailed disclosure is intended to be presented by way ofexample only and is not limiting. Various alterations, improvements, andmodifications may occur and are intended to those skilled in the art,though not expressly stated herein. These alterations, improvements, andmodifications are intended to be suggested by this disclosure, and arewithin the spirit and scope of the exemplary embodiments of thisdisclosure. For example, aspects of the present disclosure may beimplemented entirely hardware, entirely software (including firmware,resident software, micro-code, etc.) or combining software and hardwareimplementation that may all generally be referred to herein as a “unit,”“module,” or “system.”

Similarly, it should be appreciated that in the foregoing description ofembodiments of the present disclosure, various features are sometimesgrouped together in a single embodiment, figure, or description thereoffor the purpose of streamlining the disclosure aiding in theunderstanding of one or more of the various inventive embodiments. Thismethod of disclosure, however, is not to be interpreted as reflecting anintention that the claimed subject matter requires more features thanare expressly recited in each claim. Rather, inventive embodiments liein less than all features of a single foregoing disclosed embodiment.

What is claimed is:
 1. A system of computer aided diagnosis, comprising: a message server duster distributing pending data, the pending data comprising an original image; and a plurality of working nodes of a computer aided diagnosis server configured to: (1) acquire the pending data from the message server cluster, (2) segment the original image based on at least two segmentation models to determine a substance region of a candidate target and a surrounding region of the candidate target, and (3) determine the candidate target based on the substance region and the surrounding region in a streaming parallel technique in real-time, wherein the determining a candidate target by segmenting the original image based on the at least two segmentation models comprises: performing preliminary positioning on the original image to determine one or more first positioning regions, performing, based on Hessian dot-enhancement, threshold segmentation on the one or more first positioning regions to determine a second positioning region, the second positioning region including the candidate target and a background region, the candidate target including a substance region and a surrounding region that surrounds the substance region, determining the substance region of the candidate target based on the second positioning region and a statistical model, determining the surrounding region of the candidate target based on the statistical model and a morphological model, and combining the substance region and the surrounding region to determine the candidate target.
 2. The system of claim 1, wherein the message server cluster includes a Kafka cluster.
 3. The system of claim 1, wherein the pending data is processed by a Storm cluster.
 4. The system of claim 1, further comprising: a duster management node configured to manage the message server duster and the plurality of working nodes, wherein the duster management node includes a Zookeeper duster.
 5. The system of claim 1, wherein the pending data include an original medical image or intermedia data, and the intermedia data include a region of interest (ROI), preprocessed data, a candidate target, or feature data.
 6. A method of computer aided diagnosis, comprising: managing a message server duster and a plurality of working nodes of a computer aided diagnosis server to perform operations including: distributing pending data through the message server duster in a streaming mode, wherein the pending data includes an original image; acquiring the pending data from the message server duster through the plurality of working nodes of the computer aided diagnosis server; and processing the pending data in a streaming parallel mode in real time, the processing the pending data in a streaming parallel mode in real time including (1) segmenting the original image based on at least two segmentation models to determine a substance region of a candidate target and a surrounding region of the candidate target, and (2) determining the candidate target based on the substance region and the surrounding region in a streaming parallel technique in real-time, wherein the determining a candidate target by segmenting the original image based on the at least two segmentation models comprises: performing preliminary positioning on the original image to determine one or more first positioning regions, performing, based on Hessian dot-enhancement, threshold segmentation on the one or more first positioning regions to determine a second positioning region, the second positioning region including the candidate target and a background region, the candidate target including a substance region and a surrounding region that surrounds the substance region, determining the substance region of the candidate target based on the second positioning region and a statistical model, determining the surrounding region of the candidate target based on the statistical model and a morphological model, and combining the substance region and the surrounding region to determine the candidate target.
 7. A medical system, comprising: an input device configured to acquire medical data; a computer aided diagnosis device configured to acquire the medical data from the input device, the computer aided diagnosis device comprising: a message server cluster configured to acquire the medical data from the input device and distribute the medical data, the medical data comprising an original image; a plurality of working nodes of a computer aided diagnosis server configured to: (1) acquire the medical data from the message server cluster, (2) segment the original image based on at least two segmentation models to determine a substance region of a candidate target and a surrounding region of the candidate target, and (3) determine the candidate target based on the substance region and the surrounding region in a streaming parallel technique in real-time, wherein the determining a candidate target by segmenting the original image based on the at least two segmentation models comprises: performing preliminary positioning on the original image to determine one or more first positioning regions, performing, based on Hessian dot-enhancement, threshold segmentation on the one or more first positioning regions to determine a second positioning region, the second positioning region including the candidate target and a background region, the candidate target including a substance region and a surrounding region that surrounds the substance region, determining the substance region of the candidate target based on the second positioning region and a statistical model, determining the surrounding region of the candidate target based on the statistical model and a morphological model, and combining the substance region and the surrounding region to determine the candidate target; an output device configured to output the processing result.
 8. The system of claim 7, wherein the input device includes an imaging device configured to generate medical images, or a storage device configured to store data from the imaging device or data from the working nodes, the storage device includes at least one of a database, a PACS, or a file device.
 9. The system of claim 7, further comprising: a cluster management node configured to manage a Kafka cluster and the plurality of working nodes, wherein the duster management node includes a Zookeeper cluster. 